-- Logs begin at Sat 2022-09-17 20:25:12 UTC, end at Sat 2022-09-17 21:27:52 UTC. -- Sep 17 20:49:48.878726 dv-cinder-826510-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 17 20:49:48.887312 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 17 20:49:48.889983 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 17 20:49:48 2022] *** Sep 17 20:49:48.890187 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 17 20:49:48.890343 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 17 20:49:48.890484 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: nodename: dv-cinder-826510-9-iscsi Sep 17 20:49:48.890626 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: machine: x86_64 Sep 17 20:49:48.890764 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: clock source: unix Sep 17 20:49:48.890898 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: pcre jit disabled Sep 17 20:49:48.891024 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: detected number of CPU cores: 4 Sep 17 20:49:48.891163 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: current working directory: / Sep 17 20:49:48.891297 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: detected binary path: /usr/bin/uwsgi-core Sep 17 20:49:48.891439 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: your processes number limit is 47680 Sep 17 20:49:48.891567 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: your memory page size is 4096 bytes Sep 17 20:49:48.891702 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: detected max file descriptor number: 2048 Sep 17 20:49:48.891845 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: lock engine: pthread robust mutexes Sep 17 20:49:48.892128 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: thunder lock: enabled Sep 17 20:49:48.892445 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 17 20:49:48.892654 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 17 20:49:48.932165 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: Python main interpreter initialized at 0x561c31dbf9c0 Sep 17 20:49:48.932165 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: python threads support enabled Sep 17 20:49:48.932165 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: your server socket listen backlog is limited to 100 connections Sep 17 20:49:48.932165 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: your mercy for graceful operations on workers is 90 seconds Sep 17 20:49:48.932546 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: mapped 403077 bytes (393 KB) for 2 cores Sep 17 20:49:48.932617 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: *** Operational MODE: preforking *** Sep 17 20:49:48.932890 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: *** uWSGI is running in multiple interpreter mode *** Sep 17 20:49:48.932890 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: spawned uWSGI master process (pid: 81322) Sep 17 20:49:48.936867 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: spawned uWSGI worker 1 (pid: 81335, cores: 1) Sep 17 20:49:48.936867 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: spawned uWSGI worker 2 (pid: 81336, cores: 1) Sep 17 20:49:48.936867 dv-cinder-826510-9-iscsi devstack@keystone.service[81322]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 17 20:49:48.933094 dv-cinder-826510-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 17 20:49:53.256111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:49:53.256111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: from cryptography.utils import int_from_bytes Sep 17 20:49:53.256909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:49:53.256909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: from cryptography.utils import int_from_bytes Sep 17 20:49:53.260076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:49:53.260076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: from cryptography.utils import int_from_bytes Sep 17 20:49:53.260515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 17 20:49:53.260515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: from cryptography.utils import int_from_bytes Sep 17 20:49:53.307371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 20:49:53.307558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 20:49:53.307760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] command line args: [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 20:49:53.308010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 20:49:53.308188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ================================================================================ {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 20:49:53.308443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] admin_token = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.308699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] config_dir = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.308895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.309078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] config_source = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.309257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] debug = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.309456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.310645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.310645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.310645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.310836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.310987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.311168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_config_append = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.311349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.311531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_dir = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.311723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_file = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.311946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.312195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.312443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.312710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.312892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.313065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.313306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.313554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.313790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.314023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.314258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.314493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.314741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.314977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.315254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.315368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 20:49:53.315674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 20:49:53.315868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.316056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] command line args: [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 20:49:53.316219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] publish_errors = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.316379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 20:49:53.316446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.316607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ================================================================================ {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 20:49:53.316709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.316784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] admin_token = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.316849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.317662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] config_dir = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.317731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.317789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.317852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.317919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] config_source = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.317977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] standard_threads = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] debug = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] use_journal = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] use_json = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.318896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] use_stderr = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] use_syslog = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_config_append = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.319729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.319860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.320003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_dir = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.320003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_file = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.320256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.320383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.320515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.320647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.320804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.320804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.321063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.321195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.321341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.321341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.321626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.321804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.321934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.322066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.322200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.322332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.322467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.322593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.322720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.322845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.322970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.323102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.323292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.323422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.323565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.323691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.323818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.323950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.324083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.324210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.324336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.324336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.324586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.324758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.324892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] publish_errors = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.325024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.325150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.325292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.325430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.325562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.325694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.325821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.325958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.326091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.326225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.326352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.326477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] standard_threads = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.326602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.326727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.326858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.327006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.327134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.327259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.327259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] use_journal = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.327518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.327645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] use_json = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.327786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.327913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] use_stderr = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.328040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.328164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] use_syslog = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.328289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.328415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:49:53.328548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.328723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.328867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.328994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.329130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.329255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.329723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.329864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.329998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.330901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.331856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.332720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.333913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.334931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.335955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.336998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.337894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.338026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.338152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.338522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.338660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.338796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.338929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.339055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.339257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.339449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.339639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.339869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.340033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.340255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.340482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.340729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.340934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.341150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.341321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.341511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.341701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.341888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.342075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.342297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.342461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.342648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.342836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.343650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.344455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.344598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.344782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.344911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.345891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.346922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.347880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.348008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.348135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.348294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.348612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.348813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.348986 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.349980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.external = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.350909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.password = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] auth.token = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.351887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.352998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.353910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.354960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.355209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.355209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.355480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.355617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.355745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.355880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.356967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.357887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.358923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.359970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.360929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.external = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.361947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.password = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] auth.token = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.362994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.363121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.363248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.363375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.363502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.363638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.363773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.364418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.364561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.364710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.364844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.364988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.365893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.366944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.367932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.368890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.369798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.370969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.371885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.372981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.373224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.373352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.373480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.373617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.373754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.373881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.374918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.375966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.376908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.377965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.378866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.379268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.381128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.382906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.383931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] role.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.384993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] role.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.385899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.386952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.387988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.388937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.389968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.390096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.390236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.390364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.390505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.390635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.391302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.391437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.391566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.391703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.391832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.391961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.392890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.393996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.394917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.connection = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.396019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.397918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.398962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.399998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.400927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81335) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.401483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] role.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] role.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.402527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.403589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.404645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.405754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.406884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.406884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.406884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.406884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.406884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.406884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.407629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.407629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.407629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.408020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.408020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.408273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.408405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.408632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.408914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.409160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.409405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.409645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.409901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.410192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.410429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.410704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.411033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.411322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.connection = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.411605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.411861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.412109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.412356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.412596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.412879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.413150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.413396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.413649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.413891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.414141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.414382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.414641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.414882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.415132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.415367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.415610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.415850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:49:53.416022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81336) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 20:49:53.423794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.424042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.424241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.433078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.441472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.441710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.442006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.451246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.462178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.462508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.471086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.471365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.519377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.519547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.571169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.571401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.571527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.571759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.729991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.730973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.731111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.731201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.731364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.731577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.754776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.755938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.756043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.756124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.756260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.756349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.767397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.767501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.780905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.781000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.781117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.781199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.781327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81335) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.781409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.781540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81336) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 17 20:49:53.798466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.798607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.798691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:49:53.798822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:49:53.799001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.799168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.799251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:49:53.799370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 17 20:49:53.799451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.799649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.799739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.799918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.799999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.800184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.800263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.800446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.810267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.810544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.810653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.810812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.810937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.811094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.811213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.811502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.818634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.821558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.821846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.822134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.822216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.822422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.822712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.822982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.823260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.823542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.823818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.824097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.824370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.825072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.825357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.825636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.825914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.826195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.826453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.826720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.826992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.827259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.827529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.827795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.847354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.849454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.849743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.850981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.852431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.852734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.854287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.854575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.854864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.857355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.857638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.857917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.861348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.861642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.865069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.865352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.866138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.866432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.866722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.867014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.867298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.869330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.869626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.869920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.870942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.871238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.871528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.871811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.872083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.872355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.872623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.872917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.885507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.885799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.888329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.888616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.889662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.892444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.892790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.893092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.895793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.896088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.897449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.900283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.900567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.903409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.903748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.904052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.904343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.904633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.904942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.905334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.906615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.906910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.907191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.907472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.907760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.908045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.921577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.921867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.922140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.922407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.924480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.924797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.925068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.925334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.929752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.931423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.932793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.932980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.933262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.934318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.935828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.936004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.936094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.936294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.936589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.936901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.937194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.938728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.939011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.939298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.939578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.939971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.948587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.948905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.949195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.949492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.949781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.950066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.950349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.951305 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.951592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.951869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.952148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.952525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.952848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.953127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.965988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.968568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.969026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.971464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.971904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.973947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.975013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.975424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.975733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.976014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.976446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.977920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.978214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.978608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.982476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.983940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.984903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.986343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.986852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.987149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.987438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.987718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.989730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.990122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.990407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.990685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.996434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.996759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.997049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.998380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:53.998669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:53.998949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.003904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81336) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:54.004191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.004595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.004871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.005154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.005431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.005707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.005824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81335) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 17 20:49:54.006004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.006929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81336) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.007960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.008244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.008522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.008815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.009104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.009386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81335) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 17 20:49:54.044405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c31dbf9c0 pid: 81335 (default app) Sep 17 20:49:54.046884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c31dbf9c0 pid: 81336 (default app) Sep 17 20:49:54.047676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23271d-0874-49d9-ad95-7441c2a17cc2 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:49:54.047960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23271d-0874-49d9-ad95-7441c2a17cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:49:54.048233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23271d-0874-49d9-ad95-7441c2a17cc2 None None] PATH_INFO: `/v3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:49:54.052972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Sat Sep 17 20:49:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:00.789314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:00.789830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:00.789933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:00.794083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:00.799381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:00.799821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:00.799956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:00.877928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: INFO dbcounter [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] Registered counter for database keystone Sep 17 20:50:00.897633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG oslo_db.sqlalchemy.engines [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81335) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:50:00.903828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writer thread running {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 20:50:01.085234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG passlib.handlers.bcrypt [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81335) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 20:50:01.085707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG passlib.handlers.bcrypt [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81335) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 20:50:01.146569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:01.148198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:01.276327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 20:50:01.750283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:01.819742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: INFO dbcounter [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] Registered counter for database keystone Sep 17 20:50:01.841285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG oslo_db.sqlalchemy.engines [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81336) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:50:01.847387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writer thread running {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 20:50:01.966304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580a3e2-feb2-4d30-b183-c2cb73cba9ea None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:01.970588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:01.971211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:01.971493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:01.971768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:01.979841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:02.031722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING keystone.common.rbac_enforcer.enforcer [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 20:50:02.032022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c5dea-2053-4b76-9de7-c3e6b208c551 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:02.041338 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Sep 17 20:50:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 20:50:02.047241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:02.063622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bce3a-a115-4058-8f73-7fb2875b8c1f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:02.068577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:02.069711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:02.069990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:02.070269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] PATH_INFO: `/v3/projects/54beae32846044a3b8e04220f82543ad` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:02.071844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] RBAC: Authorizing `identity:get_project(project_id=54beae32846044a3b8e04220f82543ad)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:02.125064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 17 20:50:02.125407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac51fe-7b03-4273-a4cb-95f5143bbdc5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:02.129939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Sat Sep 17 20:50:02 2022] GET /identity/v3/projects/54beae32846044a3b8e04220f82543ad => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:04.263304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac6a6-8b23-4349-9a0f-17ac601885be None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:04.263842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac6a6-8b23-4349-9a0f-17ac601885be None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:04.263842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ac6a6-8b23-4349-9a0f-17ac601885be None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:04.267984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:04.273416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:04.273781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:04.273963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:04.294024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:04.295533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:04.303823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 17 20:50:04.573782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac6a6-8b23-4349-9a0f-17ac601885be None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:04.586423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ac6a6-8b23-4349-9a0f-17ac601885be None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:04.590930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b201f0-a78c-4931-b461-37c946c84619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:04.591535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b201f0-a78c-4931-b461-37c946c84619 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:04.591816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b201f0-a78c-4931-b461-37c946c84619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:04.592091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b201f0-a78c-4931-b461-37c946c84619 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:04.600695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b201f0-a78c-4931-b461-37c946c84619 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:04.602466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b201f0-a78c-4931-b461-37c946c84619 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:04.611141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Sat Sep 17 20:50:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 20:50:04.616805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:04.625747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334413f-cd97-4f17-a784-3e19cbfd9fea None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:04.630100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25620b3-bba0-41d3-b05c-52058854403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:04.630716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25620b3-bba0-41d3-b05c-52058854403a admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:04.630997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25620b3-bba0-41d3-b05c-52058854403a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:04.631273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25620b3-bba0-41d3-b05c-52058854403a admin admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:04.632616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25620b3-bba0-41d3-b05c-52058854403a admin admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:04.634467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25620b3-bba0-41d3-b05c-52058854403a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:04.636007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Sat Sep 17 20:50:04 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:06.829892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ad331-c970-4957-ac86-d4341b0bb826 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:06.830426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ad331-c970-4957-ac86-d4341b0bb826 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:06.830426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ad331-c970-4957-ac86-d4341b0bb826 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:06.831742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:06.837265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:06.837659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:06.837896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:06.855643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:06.856846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:06.880172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b6fa3-6f92-4f17-b17f-37856c870b1c None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:06.880534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b6fa3-6f92-4f17-b17f-37856c870b1c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:06.880857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b6fa3-6f92-4f17-b17f-37856c870b1c None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:06.882753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:06.887909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:06.888260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:06.888604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:06.929858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG passlib.handlers.bcrypt [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81336) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 17 20:50:06.930238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG passlib.handlers.bcrypt [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81336) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 17 20:50:06.937634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:06.980605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:06.982248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:07.080456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:07.213444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.237593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.320397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3898d8f-c5c1-4572-912c-6fdbc2b2da4e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.347455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.354916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.355550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.355826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.356096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.367711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:07.376451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:07.377415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:07.378184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:07.393450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.450997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a429967-47f4-4cd2-9b19-eee4919620e1 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.454128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.481993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.494294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.495158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.495443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.495724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.501188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6c854-2198-4106-a007-88489ce45950 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.534145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.534947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.535227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.535501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.536171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:07.536471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.538503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.550057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:07.555984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.562891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.564411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.654532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401765d-e298-4d94-9073-00d5205fba6f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.661128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:50:07.667716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.670429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.689984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.690622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.690904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.691183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.692356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.693871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.695732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:07.701050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.716312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.763163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370c52f-34d3-4a6e-9fea-72911199902c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.796425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.797502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.797774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.798047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.799931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e194c04-d31d-4185-931c-04bb6a87e4f7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.815776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.817279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.835627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:50:07.836173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:07.837431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:50:07.838753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:50:07.838753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:50:07.838753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:07.838959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.838959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.838959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.839438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:07.839520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.839604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.841094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:07.846497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.846742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:07.903155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.927681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:07.946799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f7b73-9b4a-4947-acc3-cac6c6e22826 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.949238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d1006-af4e-4dc0-a11a-b55ca889a297 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:07.979892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.980492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.980830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.981104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.981837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.domain.id=default, user.id=00fcf728ae7844bf9da557cf68143974) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:07.982148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.983577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:07.983700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6079720-92b6-4cad-8b57-724895db1c3a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.984175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:07.984455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:07.984758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:07.985399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:07.985700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:07.987066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03c035-e3f4-4ea9-a3a3-0fda4b479b50 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:07.990307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/role_assignments?user.id=00fcf728ae7844bf9da557cf68143974&scope.domain.id=default&role.id=8a4cd71b629943809347939c4c950275 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:08.005725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:10.472307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739ee29-c9ed-4576-bbc5-9e03cfc092f4 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:10.473373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739ee29-c9ed-4576-bbc5-9e03cfc092f4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:10.473871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739ee29-c9ed-4576-bbc5-9e03cfc092f4 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:10.475517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:10.482018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce628bf-1d96-449a-8510-94e5c3d0a3b1 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:10.482988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce628bf-1d96-449a-8510-94e5c3d0a3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:10.483461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce628bf-1d96-449a-8510-94e5c3d0a3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:10.511207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7ce628bf-1d96-449a-8510-94e5c3d0a3b1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:10.512829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7ce628bf-1d96-449a-8510-94e5c3d0a3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:10.606568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:10.626489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11c644-4452-4069-9b12-2592b7dd15f7 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:10.627118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11c644-4452-4069-9b12-2592b7dd15f7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:10.627591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11c644-4452-4069-9b12-2592b7dd15f7 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:10.629248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:10.635762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e5e3a-b99b-4c50-90cb-7e2df5d4d1b9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:10.636367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e5e3a-b99b-4c50-90cb-7e2df5d4d1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:10.636896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e5e3a-b99b-4c50-90cb-7e2df5d4d1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:10.669819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-819e5e3a-b99b-4c50-90cb-7e2df5d4d1b9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:10.671328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-819e5e3a-b99b-4c50-90cb-7e2df5d4d1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:10.735989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:10.767211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 12/25] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:10.774595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d4c96-5232-4efe-a330-a96e358e30c8 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:10.775142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d4c96-5232-4efe-a330-a96e358e30c8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:10.775620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d4c96-5232-4efe-a330-a96e358e30c8 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:10.777188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 14/26] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:10.782584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:10.783110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:10.783643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:10.817713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:10.819509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:10.919723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 13/27] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:11.001613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d4c96-5232-4efe-a330-a96e358e30c8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:11.116900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:11.155597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d4c96-5232-4efe-a330-a96e358e30c8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:11.241657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:11.242541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.242826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.243105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.275605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:11.284549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:11.285423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:11.286164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:11.293798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 15/28] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:11.299021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:11.306952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:11.339140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc35545-3dac-4fa5-a155-287738570ab7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:11.348158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:11.415682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:11.416326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.416603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.416902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.435711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:11.437220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:50:11.459550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:11.460635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:50:11.461857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:50:11.461857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:50:11.461857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:11.461857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 14/29] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:11.469083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:11.528116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe8345-029b-4b96-ada0-5433197c269a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:11.563899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:11.564891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.565393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.565860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.566720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:11.567213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:11.569672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7797f7-5c3c-43d7-8c69-aef493cef026 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:11.578416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 16/30] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:11.580291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd49d-64b6-4125-91ea-01948ac94510 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.580813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd49d-64b6-4125-91ea-01948ac94510 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.581297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd49d-64b6-4125-91ea-01948ac94510 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.583070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 17/31] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:11.583954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd49d-64b6-4125-91ea-01948ac94510 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:11.598428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bce2b6-7658-45a0-bd63-daea5e1e7420 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:11.640787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:11.641924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.642380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.643120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.643946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:11.644433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:11.646046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81beb5-718e-4664-a2d9-a7be8680db3d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:11.659698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 15/32] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:11.661376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.661822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.662260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.673382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:11.686286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:11.687954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:11.705722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:11.722466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd49d-64b6-4125-91ea-01948ac94510 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:11.767744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:11.768455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:11.768828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:11.769177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:11.770163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:11.772362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:11.776068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 16/33] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:11.777386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:11.963254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af9fe-0645-4305-ae44-b6aa2b767f7d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:11.965845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 18/34] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 17 20:50:11 2022] POST /identity/v3/roles => generated 219 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:11.966606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:12.001716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:12.002374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.002654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.002927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.004489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:12.005975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:12.008272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 17/35] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:11 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:12.013919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:12.034043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:12.145361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306534-e576-4ecd-9178-2153d65ee8c7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:12.186652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:12.195340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b9afc-ec5a-4154-8fed-ff1d8ef6de9c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:12.205815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:12.206632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.208080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.208413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.210311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:12.215489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:12.277304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 19/36] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:50:11 2022] POST /identity/v3/projects => generated 301 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:12.277978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:12.278629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:12.278738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.278926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.279202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.280433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:12.282039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:12.283655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 18/37] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:12.293527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:12.431273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:12.445534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abe0f-bb37-4fd2-b074-f7c4afc33c92 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:12.451520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3572f88-13a7-48f0-8c9f-8b4e9821f35a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:12.488740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559322b-151f-432f-a88a-62564339ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:12.489298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559322b-151f-432f-a88a-62564339ff9e None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.489586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559322b-151f-432f-a88a-62564339ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.489867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559322b-151f-432f-a88a-62564339ff9e None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.490562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559322b-151f-432f-a88a-62564339ff9e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:12.492566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96ea78-694e-4d29-9961-44c9d3420708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:12.493115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559322b-151f-432f-a88a-62564339ff9e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:12.493304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ea78-694e-4d29-9961-44c9d3420708 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.493569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ea78-694e-4d29-9961-44c9d3420708 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.493896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96ea78-694e-4d29-9961-44c9d3420708 None admin] PATH_INFO: `/v3/domains/default/users/00fcf728ae7844bf9da557cf68143974/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.495957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96ea78-694e-4d29-9961-44c9d3420708 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=00fcf728ae7844bf9da557cf68143974, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:12.497486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96ea78-694e-4d29-9961-44c9d3420708 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:12.509236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 20/38] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 17 20:50:12 2022] POST /identity/v3/roles => generated 225 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:12.510720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb322-0014-435f-9dec-8e54bfe75b58 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.511000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb322-0014-435f-9dec-8e54bfe75b58 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.511280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb322-0014-435f-9dec-8e54bfe75b58 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.517175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 21/39] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:12.517175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f40ca1-0cc5-4b40-a57c-0f5904e6c293 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.517175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f40ca1-0cc5-4b40-a57c-0f5904e6c293 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.517175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f40ca1-0cc5-4b40-a57c-0f5904e6c293 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.517175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 22/40] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:12.521425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.521603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.521952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.545645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:12.547235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:12.601114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:12.640081 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 19/41] 10.222.0.16 () {58 vars in 1361 bytes} [Sat Sep 17 20:50:12 2022] PUT /identity/v3/domains/default/users/00fcf728ae7844bf9da557cf68143974/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:12.641967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:12.642268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:12.642548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:12.663413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 23/42] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:12.669305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:12.671418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:12.754738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 20/43] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:12.933230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:13.031034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd683ec-8e95-4a0d-9402-4a441c1d5eb2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:13.031545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:13.068020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:13.068622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:13.069285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:13.069656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:13.070331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:13.071787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:13.089653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 24/44] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 17 20:50:12 2022] POST /identity/v3/roles => generated 223 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:13.114237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:13.147027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972df13-612a-47a0-a264-fd257620c8f9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:13.183479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075fb80-6796-4083-ae16-8c08ba09b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:13.184139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fb80-6796-4083-ae16-8c08ba09b194 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:13.184419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fb80-6796-4083-ae16-8c08ba09b194 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:13.184718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075fb80-6796-4083-ae16-8c08ba09b194 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:13.185890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fb80-6796-4083-ae16-8c08ba09b194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:13.187578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075fb80-6796-4083-ae16-8c08ba09b194 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:13.192342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 21/45] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:13.198340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:13.230496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:13.312887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf7ad9-30cf-4cae-ae15-6e036b8e0d50 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:13.348352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4047a-311c-4d8b-98cc-77f0c5959355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:13.348987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4047a-311c-4d8b-98cc-77f0c5959355 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:13.349266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4047a-311c-4d8b-98cc-77f0c5959355 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:13.349539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4047a-311c-4d8b-98cc-77f0c5959355 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:13.350487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4047a-311c-4d8b-98cc-77f0c5959355 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:13.351933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4047a-311c-4d8b-98cc-77f0c5959355 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:13.483545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 25/46] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:50:13 2022] POST /identity/v3/projects => generated 312 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:14.828883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba46dcd-7f6d-412f-a66a-f0403bbec4b7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:14.829431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba46dcd-7f6d-412f-a66a-f0403bbec4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:14.829431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba46dcd-7f6d-412f-a66a-f0403bbec4b7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:14.831048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 22/47] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:14.836436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:14.836861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:14.837020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:14.855095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:14.856282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:14.920485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 26/48] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:15.188510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba46dcd-7f6d-412f-a66a-f0403bbec4b7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:15.288411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba46dcd-7f6d-412f-a66a-f0403bbec4b7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:15.320021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:15.320643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.320953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.321224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.332556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:15.338933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:15.339743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:15.340481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 23/49] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:15.347165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:15.359167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:15.450752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47f08b-d3aa-4c63-abfc-cbedb486a85e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:15.485125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:15.485754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.486036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.486313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.486979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:15.487283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:15.488655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:15.497105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 27/50] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:15.502898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:15.505919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:15.606384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a35292-f6a2-4697-a5ba-baed6c4f4206 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:15.640116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b7353-5b6e-4847-89a0-5d4e9141ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:15.640769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7353-5b6e-4847-89a0-5d4e9141ab10 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.641056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7353-5b6e-4847-89a0-5d4e9141ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.641340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b7353-5b6e-4847-89a0-5d4e9141ab10 None admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.642469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7353-5b6e-4847-89a0-5d4e9141ab10 None admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:15.643918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b7353-5b6e-4847-89a0-5d4e9141ab10 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:15.645541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 24/51] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:15 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:15.651902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:15.695058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62053ab5-79e7-4790-b7bf-adb47b63ea3e None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.695344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62053ab5-79e7-4790-b7bf-adb47b63ea3e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.695630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62053ab5-79e7-4790-b7bf-adb47b63ea3e None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.697009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 25/52] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:15.701955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.702182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.702457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.724011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:15.725096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:15.726568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:15.754066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd90ce-28c7-4b24-9d6c-ae1b62622e83 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:15.786892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:15.787581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.787799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.788075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.789203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:15.789782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:15.790689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:15.792433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 28/53] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:15.796814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 26/54] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:15.798065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:15.894609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7382e77-0372-43e3-8223-7ca35d78465f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:15.926937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:15.927548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:15.927829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:15.928631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:15.929397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.domain.id=default, user.id=00fcf728ae7844bf9da557cf68143974) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:15.929700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:15.931154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:15.937473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 29/55] 10.222.0.16 () {56 vars in 1322 bytes} [Sat Sep 17 20:50:15 2022] GET /identity/v3/role_assignments?user.id=00fcf728ae7844bf9da557cf68143974&scope.domain.id=default&role.id=8a4cd71b629943809347939c4c950275 => generated 530 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:16.082396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:16.297193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1cb0c-2cb5-46f4-8ba6-37e48d9bb5a5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:16.331715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509649-033b-4230-837b-5192668af454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:16.332344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509649-033b-4230-837b-5192668af454 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:16.332704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509649-033b-4230-837b-5192668af454 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:16.332940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509649-033b-4230-837b-5192668af454 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:16.334048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509649-033b-4230-837b-5192668af454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:16.335752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509649-033b-4230-837b-5192668af454 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:16.337369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 27/56] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:16.342975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:16.346685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:16.442572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44487e2-23e0-4365-83d1-5a974606689e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:16.474637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ebbd2-1dd2-4706-bac9-46d3ae46b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:16.475747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ebbd2-1dd2-4706-bac9-46d3ae46b03b None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:16.476024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ebbd2-1dd2-4706-bac9-46d3ae46b03b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:16.476300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ebbd2-1dd2-4706-bac9-46d3ae46b03b None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:16.476986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ebbd2-1dd2-4706-bac9-46d3ae46b03b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:16.478445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ebbd2-1dd2-4706-bac9-46d3ae46b03b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:16.505065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 30/57] 10.222.0.16 () {60 vars in 1140 bytes} [Sat Sep 17 20:50:16 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:18.665518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3b03-ea75-4209-9183-487ff4c6a0af None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:18.666001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3b03-ea75-4209-9183-487ff4c6a0af None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:18.666001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3b03-ea75-4209-9183-487ff4c6a0af None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:18.667313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 28/58] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:18.672493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:18.672889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:18.673089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:18.691518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:18.692739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:18.742584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:18.759552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 31/59] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:19.029120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3b03-ea75-4209-9183-487ff4c6a0af None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:19.095250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:19.129444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3b03-ea75-4209-9183-487ff4c6a0af None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:19.161211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00642e5b-9e18-4df3-b0e2-b127cf52475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:19.161829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00642e5b-9e18-4df3-b0e2-b127cf52475a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:19.162112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00642e5b-9e18-4df3-b0e2-b127cf52475a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:19.162903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00642e5b-9e18-4df3-b0e2-b127cf52475a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:19.164014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00642e5b-9e18-4df3-b0e2-b127cf52475a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:19.165476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00642e5b-9e18-4df3-b0e2-b127cf52475a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:19.167068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 29/60] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:19.172610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:19.275402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb8b1-f095-4e76-be53-057be6255482 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:19.309228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41ddc8-3c7b-4cdc-a77e-afaaf9a5f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:19.309770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41ddc8-3c7b-4cdc-a77e-afaaf9a5f606 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:19.310048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41ddc8-3c7b-4cdc-a77e-afaaf9a5f606 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:19.310319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41ddc8-3c7b-4cdc-a77e-afaaf9a5f606 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:19.310957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41ddc8-3c7b-4cdc-a77e-afaaf9a5f606 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:19.312404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41ddc8-3c7b-4cdc-a77e-afaaf9a5f606 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:19.337979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 32/61] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:19 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:22.527607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae7601-347a-4dcc-b917-7bd3b1dcca62 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.528707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae7601-347a-4dcc-b917-7bd3b1dcca62 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.529229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae7601-347a-4dcc-b917-7bd3b1dcca62 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:22.530907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 30/62] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:22.537190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2522e-278f-4b64-8372-c5533d588350 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.538576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2522e-278f-4b64-8372-c5533d588350 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.539110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2522e-278f-4b64-8372-c5533d588350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:22.570267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1ef2522e-278f-4b64-8372-c5533d588350 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:22.571811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1ef2522e-278f-4b64-8372-c5533d588350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:22.619321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:22.666101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 33/63] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:22.720988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36fa74-77e3-49ff-a076-ed35090e7bc3 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.721307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36fa74-77e3-49ff-a076-ed35090e7bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.721556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36fa74-77e3-49ff-a076-ed35090e7bc3 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:22.722919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 31/64] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:22.729236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b91148-6196-49f2-aa85-7ddf95ba146d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.729532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b91148-6196-49f2-aa85-7ddf95ba146d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.729878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b91148-6196-49f2-aa85-7ddf95ba146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:22.757976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c8b91148-6196-49f2-aa85-7ddf95ba146d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:22.759197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c8b91148-6196-49f2-aa85-7ddf95ba146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:22.858862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 34/65] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:22.967547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d7754-9622-4d95-93af-74020ed98ea6 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.967862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d7754-9622-4d95-93af-74020ed98ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.968107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d7754-9622-4d95-93af-74020ed98ea6 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:22.969789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 32/66] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:22.975714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021540-9def-405e-83ba-ff245a5c4c8b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:22.976220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021540-9def-405e-83ba-ff245a5c4c8b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:22.976512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3021540-9def-405e-83ba-ff245a5c4c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.001921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a3021540-9def-405e-83ba-ff245a5c4c8b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.003252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a3021540-9def-405e-83ba-ff245a5c4c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.090236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 35/67] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:23.139313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d7754-9622-4d95-93af-74020ed98ea6 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.155861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6c0f-54c1-4a86-9daa-d7c7a59e4c35 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.156150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6c0f-54c1-4a86-9daa-d7c7a59e4c35 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.156427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a6c0f-54c1-4a86-9daa-d7c7a59e4c35 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.157813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 36/68] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:23.163441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.163959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.164243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.197265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.198487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.247439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.316931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d7754-9622-4d95-93af-74020ed98ea6 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.320943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 37/69] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 17 20:50:23.321899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.363598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.364241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.364529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.364841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.368087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.386302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:23.400072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:23.400997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:23.401754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 33/70] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:23.402773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.462261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5310a-b43b-46c2-b5cc-020ebfa3d4d2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.499094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.499751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.500093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.500368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.508402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ee008-5d19-47b5-84db-433072e38252 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.511251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:23.515319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:23.516143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 38/71] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:23.517272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.543469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.550710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c3c94-547f-4ea8-b802-9027752a5a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.551652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3c94-547f-4ea8-b802-9027752a5a99 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.552081 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3c94-547f-4ea8-b802-9027752a5a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.552358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c3c94-547f-4ea8-b802-9027752a5a99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.562936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c3c94-547f-4ea8-b802-9027752a5a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0d1c3c94-547f-4ea8-b802-9027752a5a99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:23.567705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:23.568988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:23.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 34/72] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:23.574575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f02226-8bb9-4fc4-9ab7-50042e34d9a5 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.574868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f02226-8bb9-4fc4-9ab7-50042e34d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.575196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f02226-8bb9-4fc4-9ab7-50042e34d9a5 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.578307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 35/73] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:23.578865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f02226-8bb9-4fc4-9ab7-50042e34d9a5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.632745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b679c-5556-4c47-860f-30a767e8f89f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.637816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.666962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.667570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.667850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.668127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.668830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:23.669132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.670532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.678934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 39/74] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:23.679545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.681212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f02226-8bb9-4fc4-9ab7-50042e34d9a5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.715442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.716049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.716331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.716606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.717292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:23.717594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.718977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f566830-8cf8-4aba-9dc3-775fe3fd2da0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.735105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 36/75] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:23.735105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.735105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.735105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.750285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:23.751390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:23.762028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.778327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10237b-8b9b-4f55-88c8-fdae8f2417ba None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.811643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.812238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.812526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.812830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.813485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:23.813782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.815163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.816819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 37/76] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:23.817654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.823613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 40/77] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:23.828321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.871389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:23.926216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57734c73-084d-4175-89a7-f51733e7b684 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.935700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe9762-3b82-4647-a8b9-88d8eaed7434 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:23.962003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.962631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.962911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.963182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.971106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:23.971736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:23.972020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:23.972300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:23.973590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:23.978779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:23.979602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 38/78] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:23.981447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:23.992167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:23.993942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:23.995698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 41/79] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:23.996525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.000770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.091297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd1690-fb1b-47aa-977b-d603ba2adc88 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.100476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.102064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5940ff3e-75fe-4c13-8f2a-100b10ba9353 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.125752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.126380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.126697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.126975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.128107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.129594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.131743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 39/80] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.132506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.137886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.138517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.138799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.139073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.140201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.141688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.143261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 42/81] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:23 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.144017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.214376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.235873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0507843-134d-484d-b2db-d10c93e1b79f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.243632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7c6f8-7ccd-4989-83ec-2e650a716e2b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.270495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.271027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.271308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.271612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.272268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:24.272567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.273972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.277246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.277874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.278152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.278435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.281676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 40/82] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.282469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.320390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.344683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.346181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.378241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ea0a8-02d2-4726-8689-98dd64a6cbe3 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.414302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.416057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.416771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.417062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.417409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.418551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.420011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.421622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 41/83] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.422384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.450540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 43/84] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:24.451389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.518607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f536be-d16f-4708-8940-d6189d394af8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.525528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.549981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba95a1-83a6-4212-9917-fefb993b7180 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.554675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.555283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.555569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.555849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.583397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.583994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.584720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.584964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.622026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.623478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.624544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.651163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.652620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.718476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 42/85] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:24.719332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.729334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.749178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 44/86] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:24.750077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.816957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.820274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7030d5-161e-48cd-ae60-629b82ffd898 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.846321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081fa10-c06e-4ac8-a080-6089b626cf85 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:24.855483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.856116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.856400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.856699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.857837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.859295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.860917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 43/87] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:24.861680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.884442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:24.885154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:24.885434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:24.885704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:24.886346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:24.887789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:24.909280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 45/88] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:24 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:24.910133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:24.941683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:24.973194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a01c1-ce3b-4c75-90b3-3dfaabc365ce None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.014467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.015126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.015415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.015685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.016912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.018422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.020107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 44/89] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.025973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.043972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f07f7-3252-4c0d-93a0-09184ebf6adc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.072136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.082211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.083597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.083597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.083597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.085199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.086671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.088552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 46/90] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:24 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.089132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.152006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c5880-22fc-4360-ac3b-7f3aa91e8c5c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.198973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.199974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.199974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.199974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.202324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.203781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.206012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 45/91] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.206572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.211393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad827c-0469-472e-9201-5812e1eda142 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.229215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.258811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.259532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.259812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.260084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.338172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427be987-cb94-451a-9210-b734a0470c94 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.344982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.346468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.372716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.375611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.376233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.376519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.376836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.377605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=e925dc6d06704d19b20429d1596b7d2d) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:25.377919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.379370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.393169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 46/92] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/role_assignments?user.id=e925dc6d06704d19b20429d1596b7d2d&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.394045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.481279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 47/93] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:25.482176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.539742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.587765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e62a75-b0c3-494c-b049-05ebacf481da None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.592402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:25.600022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecccb0e-a80c-437a-9f10-e02babf14267 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.639921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.640526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.640838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.641115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.641854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] RBAC: Adding query filter params (role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=e925dc6d06704d19b20429d1596b7d2d) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:25.642156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.643606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.649428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 48/94] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/role_assignments?user.id=e925dc6d06704d19b20429d1596b7d2d&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.650259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.658694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.659329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.659618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.659900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.660652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=00fcf728ae7844bf9da557cf68143974) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:25.661072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.662933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.679471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 47/95] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/role_assignments?user.id=00fcf728ae7844bf9da557cf68143974&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.762979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ed3a3-e339-40e3-83e8-1499a1504133 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:25.800979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c40ba8-e155-4953-ba60-84c279e61e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:25.802130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c40ba8-e155-4953-ba60-84c279e61e77 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:25.802594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c40ba8-e155-4953-ba60-84c279e61e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:25.803645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c40ba8-e155-4953-ba60-84c279e61e77 None admin] PATH_INFO: `/v3/projects/dc17fc371eda4a3285526962b0406315` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:25.805404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c40ba8-e155-4953-ba60-84c279e61e77 None admin] RBAC: Authorizing `identity:get_project(project_id=dc17fc371eda4a3285526962b0406315)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:25.806931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c40ba8-e155-4953-ba60-84c279e61e77 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:25.808662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 49/96] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/projects/dc17fc371eda4a3285526962b0406315 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:25.817921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:25.979288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670f4c6-2416-4ed4-9204-e602a86a604a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:26.026150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:26.026771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:26.027059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:26.027342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:26.028146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=dc17fc371eda4a3285526962b0406315, user.id=e925dc6d06704d19b20429d1596b7d2d) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:26.028499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:26.030009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d24a0-8a4b-40b6-844a-7874f8870837 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:26.042083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 48/97] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:25 2022] GET /identity/v3/role_assignments?user.id=e925dc6d06704d19b20429d1596b7d2d&scope.project.id=dc17fc371eda4a3285526962b0406315&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:27.842281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea756c5-9754-482b-af03-4a46f8e80066 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.842802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea756c5-9754-482b-af03-4a46f8e80066 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.842802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea756c5-9754-482b-af03-4a46f8e80066 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.844399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 50/98] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:27.851466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c587e2-044d-4b29-b9ec-3858e3a5cd32 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:27.852407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c587e2-044d-4b29-b9ec-3858e3a5cd32 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:27.852921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c587e2-044d-4b29-b9ec-3858e3a5cd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:27.859636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:27.882196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-44c587e2-044d-4b29-b9ec-3858e3a5cd32 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:27.884064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-44c587e2-044d-4b29-b9ec-3858e3a5cd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:27.971958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 49/99] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:28.002377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664555d-1687-4c6c-bdf6-a212d70e68d3 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.002989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664555d-1687-4c6c-bdf6-a212d70e68d3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.003516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664555d-1687-4c6c-bdf6-a212d70e68d3 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.005335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 51/100] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:28.010292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becccf2c-2ca8-4150-aa80-fe012b50f7dd None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.010796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becccf2c-2ca8-4150-aa80-fe012b50f7dd None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.011284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becccf2c-2ca8-4150-aa80-fe012b50f7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.041597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-becccf2c-2ca8-4150-aa80-fe012b50f7dd None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.043121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-becccf2c-2ca8-4150-aa80-fe012b50f7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.127206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 50/101] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:28.288959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd635cf9-b5dc-4ead-9b90-2e34b8293203 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.289697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd635cf9-b5dc-4ead-9b90-2e34b8293203 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.290182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd635cf9-b5dc-4ead-9b90-2e34b8293203 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.291708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 52/102] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:28.299389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.299892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.300385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.329884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:28.331389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:28.404702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.411905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd635cf9-b5dc-4ead-9b90-2e34b8293203 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.418178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 51/103] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:28.467156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.565380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd635cf9-b5dc-4ead-9b90-2e34b8293203 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.611766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.612827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.613371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.613833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.629879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:28.635543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:28.636364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:28.637123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:28.639611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 53/104] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:28.645393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.692213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.767559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94649a-ce4a-4529-b16b-80ad1e5bc98c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.826474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.827360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.827792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.828371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.829558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:28.830260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.831777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.832503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:28.850923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 52/105] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:28.852934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.858478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4c8fd-042f-435f-bee8-bf095c023389 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:28.925414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:28.926390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:28.926859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:28.929587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:28.932512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:28.934195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:28.936029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 54/106] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:28.938192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:28.967560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.064786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f71303-c1b6-42df-94fd-c1b062dd8d06 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.130606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.131260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.131546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.131824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.138861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a68c9-73ae-4970-8df6-7240d48f2d68 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.143851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:29.150073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:29.150898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:29.151667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:29.152267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 53/107] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:29.152896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.180330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf86a5-e1f1-4a4a-bd81-f6973e903499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.180908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf86a5-e1f1-4a4a-bd81-f6973e903499 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.181290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf86a5-e1f1-4a4a-bd81-f6973e903499 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.181566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf86a5-e1f1-4a4a-bd81-f6973e903499 None admin] PATH_INFO: `/v3/users/231ef980cbff45db85b8bdcef0e552cd` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.195866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf86a5-e1f1-4a4a-bd81-f6973e903499 None admin] RBAC: Authorizing `identity:get_user(user_id=231ef980cbff45db85b8bdcef0e552cd)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.197825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf86a5-e1f1-4a4a-bd81-f6973e903499 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.204238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 55/108] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:28 2022] GET /identity/v3/users/231ef980cbff45db85b8bdcef0e552cd => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:29.204238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8005f8-81b7-45e8-b39c-8678581be4e9 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.204238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8005f8-81b7-45e8-b39c-8678581be4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.204238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8005f8-81b7-45e8-b39c-8678581be4e9 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.204238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 56/109] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:29.204238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394b644-0a2e-47b9-96bb-d287a3f14a44 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.204919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394b644-0a2e-47b9-96bb-d287a3f14a44 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.204919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394b644-0a2e-47b9-96bb-d287a3f14a44 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.206686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 57/110] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:29.206830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394b644-0a2e-47b9-96bb-d287a3f14a44 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.222388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.270462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2cd61-bc09-489a-b749-be5f1be08a4b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.287948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.311953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.312564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.313646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.313927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.314572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.316018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.320226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394b644-0a2e-47b9-96bb-d287a3f14a44 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.347487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 54/111] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:29.348408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.375634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.376200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.376485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.376792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.377526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:29.378090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.379249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989400c-96d3-4bcf-8577-ddea3acd1f7e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.394384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 58/112] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:29.401157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593f11-d5da-411c-bc5d-212b5d8fabc8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.401453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593f11-d5da-411c-bc5d-212b5d8fabc8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.401732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e593f11-d5da-411c-bc5d-212b5d8fabc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.430432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0e593f11-d5da-411c-bc5d-212b5d8fabc8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.432031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0e593f11-d5da-411c-bc5d-212b5d8fabc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.488251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4dab6-b047-4104-beea-10d0ef19b407 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.518447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 59/113] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:29.519615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.520214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.520214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.529851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c694513-d0cb-4de7-977f-ba41ec0c3417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.530472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c694513-d0cb-4de7-977f-ba41ec0c3417 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.530754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c694513-d0cb-4de7-977f-ba41ec0c3417 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.531030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c694513-d0cb-4de7-977f-ba41ec0c3417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.543754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.545462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.548984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.569945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.617707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c694513-d0cb-4de7-977f-ba41ec0c3417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.619440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c694513-d0cb-4de7-977f-ba41ec0c3417 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.643396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 60/114] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:29.645452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.782655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 55/115] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:29.784319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d09b62-4854-49fb-9f27-7b83fcca52fb None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.784862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d09b62-4854-49fb-9f27-7b83fcca52fb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.785328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d09b62-4854-49fb-9f27-7b83fcca52fb None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.787621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 56/116] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:29.792038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d09b62-4854-49fb-9f27-7b83fcca52fb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.828323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e755e-a2f8-49e1-90f4-74e8e741b53d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.872865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.875304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f33169-b179-4270-a489-3afad4b7ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.875831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f33169-b179-4270-a489-3afad4b7ea74 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.876124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f33169-b179-4270-a489-3afad4b7ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.876364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f33169-b179-4270-a489-3afad4b7ea74 None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.877533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f33169-b179-4270-a489-3afad4b7ea74 None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.878996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f33169-b179-4270-a489-3afad4b7ea74 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.881476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 61/117] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:29.882413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121415-9f02-4657-991b-590a157f281d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.882699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121415-9f02-4657-991b-590a157f281d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.882965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87121415-9f02-4657-991b-590a157f281d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.895671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:29.908800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-87121415-9f02-4657-991b-590a157f281d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:29.909992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-87121415-9f02-4657-991b-590a157f281d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:29.927427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d09b62-4854-49fb-9f27-7b83fcca52fb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:29.975482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:29.976607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:29.977429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:29.978181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:29.979682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:29.981854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:29.984005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 57/118] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:29.984983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:29.985102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 62/119] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:29.993056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121415-9f02-4657-991b-590a157f281d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.112652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d4c11-019f-48ad-8abc-a79f9d218322 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.113131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87121415-9f02-4657-991b-590a157f281d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.149243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.149871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.150149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.150422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.151531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.153028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.154368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.154649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 63/120] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:30.155410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.155622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.156117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.156579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.158085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.171195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.173600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:30.181205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:30.182056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:30.182817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:30.185304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 58/121] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:30.186286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.270004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dafd5-f8a2-45ad-b2b2-a372c1f26cdc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.303669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726784f-2d3d-4584-8e34-a6e2c6435b04 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.305936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.306935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.306935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.307096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.307794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] RBAC: Adding query filter params (role.id=1c84e2fcb51c4227a0edc690959c2fb1, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=231ef980cbff45db85b8bdcef0e552cd) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:30.308100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.309591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.315886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 64/122] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/role_assignments?user.id=231ef980cbff45db85b8bdcef0e552cd&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=1c84e2fcb51c4227a0edc690959c2fb1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:30.316844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.350319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.352253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.352813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.353488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.369545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:30.376240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:30.377264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:30.378083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:30.381131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 59/123] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:30.386547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.411779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.453335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.461878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329d891-a745-4aa8-be9b-a71ffc5d5e22 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.503355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.504368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.505344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/e925dc6d06704d19b20429d1596b7d2d/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.507012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110b483-8f80-48cd-9e8b-06e96f77076f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.507804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] RBAC: Authorizing `identity:create_grant(project_id=530e3f52da9144b3b7b0c97df02943f7, user_id=e925dc6d06704d19b20429d1596b7d2d, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.509504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.526467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 65/124] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:30 2022] PUT /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/e925dc6d06704d19b20429d1596b7d2d/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:30.527679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.544916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.545618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.545969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.546314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.547057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:30.547424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.549051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.558816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 60/125] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:30.559773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.647931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f5d76-c883-4ec0-b0e1-c6e4385cb93d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.666540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ce9e8-1d34-49da-88b8-15f6346d5c4e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.678519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.684581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.692494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.693881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.694512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.695104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.707950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:30.708242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.708908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.709236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.709449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.710122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:30.710422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.711798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:30.715727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:30.716545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:30.717337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:30.719871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 66/126] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:30.720245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 61/127] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:30.725775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.727248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.838342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377e7cc-859c-4908-8309-3fdd7cc4db21 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.840214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518339-a663-4ae1-a8ec-878d42022d78 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:30.875158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.875825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.876083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.876424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.877869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.878626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:30.879347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.879942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:30.880408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:30.880979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:30.883638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:30.883926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.884704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:30.886478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:30.888405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 67/129] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:30.889404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:30.939486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:30.953166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.002502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb36862-2fd6-4653-a26c-892066c42772 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.003667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd02ab-9c89-4bc3-bedf-a2e8b7ae77e4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.040218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.041265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.041735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.042207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.042481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.043111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.043482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.043768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.044310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.044473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:31.044766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.046166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.046519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.048573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 68/130] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:31.049567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.055450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 63/131] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:31.056131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.164346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f603-0d2e-4bfa-863a-aad5970eea93 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.194188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.199678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e812e-f8bc-475b-b69b-3a4f01bad622 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.203181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.204075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.204547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.205287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.211454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.213169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.216527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 69/132] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:31 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:31.218327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.223210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.292564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.293034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.293314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.293595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/00fcf728ae7844bf9da557cf68143974/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.301565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] RBAC: Authorizing `identity:create_grant(project_id=530e3f52da9144b3b7b0c97df02943f7, user_id=00fcf728ae7844bf9da557cf68143974, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.304221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.354440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 64/133] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:31 2022] PUT /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/00fcf728ae7844bf9da557cf68143974/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.355278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.373944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb3a0e-cf99-4786-88a3-c30f7a6b0973 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.418400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.419271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.419554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.419874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.421539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.423910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.425775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 70/134] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:31 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:31.432964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.506954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.557918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba3d68-067c-421c-890d-0635e425c2a2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.569732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d260-ece3-491c-8af6-7f0b3ded444a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.595721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.596451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.596772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.597142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] PATH_INFO: `/v3/projects/dc17fc371eda4a3285526962b0406315` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.598269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] RBAC: Authorizing `identity:get_project(project_id=dc17fc371eda4a3285526962b0406315)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.600183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.604868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18c3f6-ef73-4c70-a86a-18b1020fe76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.605405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 71/135] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:31 2022] GET /identity/v3/projects/dc17fc371eda4a3285526962b0406315 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:31.605569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18c3f6-ef73-4c70-a86a-18b1020fe76c None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.605824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18c3f6-ef73-4c70-a86a-18b1020fe76c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.606102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18c3f6-ef73-4c70-a86a-18b1020fe76c None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/e925dc6d06704d19b20429d1596b7d2d/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.608432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18c3f6-ef73-4c70-a86a-18b1020fe76c None admin] RBAC: Authorizing `identity:create_grant(project_id=530e3f52da9144b3b7b0c97df02943f7, user_id=e925dc6d06704d19b20429d1596b7d2d, role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.609967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18c3f6-ef73-4c70-a86a-18b1020fe76c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.612410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:31.626353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 65/136] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:31 2022] PUT /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/e925dc6d06704d19b20429d1596b7d2d/roles/44957acdbea44978962b0cc8d2e0d069 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:31.738563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148d885-4349-4f65-b47c-4a1647129b91 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:31.773280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.783762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe17ea-8e2e-470d-95df-2e57153359e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:31.784461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe17ea-8e2e-470d-95df-2e57153359e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:31.784815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe17ea-8e2e-470d-95df-2e57153359e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:31.785084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe17ea-8e2e-470d-95df-2e57153359e6 None admin] PATH_INFO: `/v3/projects/dc17fc371eda4a3285526962b0406315/users/e925dc6d06704d19b20429d1596b7d2d/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:31.787311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe17ea-8e2e-470d-95df-2e57153359e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc17fc371eda4a3285526962b0406315, user_id=e925dc6d06704d19b20429d1596b7d2d, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:31.788918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe17ea-8e2e-470d-95df-2e57153359e6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:31.805970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 72/137] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:31 2022] PUT /identity/v3/projects/dc17fc371eda4a3285526962b0406315/users/e925dc6d06704d19b20429d1596b7d2d/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:32.762508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f257f-33fb-4811-aa86-539e791e14b1 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.763052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f257f-33fb-4811-aa86-539e791e14b1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.763052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f257f-33fb-4811-aa86-539e791e14b1 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.764405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 66/138] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:50:32.782375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3ae9d-8922-4fc0-b2da-9dd3dd534f6c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:32.783341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3ae9d-8922-4fc0-b2da-9dd3dd534f6c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:32.783831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3ae9d-8922-4fc0-b2da-9dd3dd534f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:32.809699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5de3ae9d-8922-4fc0-b2da-9dd3dd534f6c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:32.811306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5de3ae9d-8922-4fc0-b2da-9dd3dd534f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:32.903094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 73/139] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:33.105799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06d855-8c8d-4752-9eaa-73bd04fb89f3 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.105989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06d855-8c8d-4752-9eaa-73bd04fb89f3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.106260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06d855-8c8d-4752-9eaa-73bd04fb89f3 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.107913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 67/140] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:33.113378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.113665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.113943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.141697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:33.142904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:33.223244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 74/141] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:33.407252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06d855-8c8d-4752-9eaa-73bd04fb89f3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.449837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.558682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06d855-8c8d-4752-9eaa-73bd04fb89f3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.609458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.610507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.610993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.611478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.612905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.614566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.617440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 68/142] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:33.622873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.715134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:33.792982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bfb86-17c0-41b5-a982-baf5b1242412 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:33.837337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:33.837696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:33.837871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:33.838146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:33.838802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:33.840257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:33.860719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:33.908338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 75/143] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:33 2022] POST /identity/v3/users => generated 283 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:34.029701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.035645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f80bb7-8204-435d-a937-5e516a17276e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.088366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52def9-4d14-41bf-ab93-c99a32f31922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.089048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52def9-4d14-41bf-ab93-c99a32f31922 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.089336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52def9-4d14-41bf-ab93-c99a32f31922 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.089616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52def9-4d14-41bf-ab93-c99a32f31922 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.109239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52def9-4d14-41bf-ab93-c99a32f31922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5a52def9-4d14-41bf-ab93-c99a32f31922 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:34.114599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:34.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:34.118159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 69/144] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:34.125706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.148967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efdd7e-4f95-4f8c-a78b-789a8b877ce7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.149300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efdd7e-4f95-4f8c-a78b-789a8b877ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.149534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efdd7e-4f95-4f8c-a78b-789a8b877ce7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.150903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 70/145] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:34.164123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.164691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.165120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.216235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:34.217569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.328828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 71/146] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:34.360467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd2a1-501c-4502-9090-d47856e32b87 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.362229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.421948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.422574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.422860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.423138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.423813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:34.424115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.425536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.436424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 76/147] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:34.442372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.555072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.573083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a50cf8-9e2b-465e-84c8-a10ece59e796 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.618604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.619156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.619437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.619711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] PATH_INFO: `/v3/users/231ef980cbff45db85b8bdcef0e552cd` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.622491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] RBAC: Authorizing `identity:get_user(user_id=231ef980cbff45db85b8bdcef0e552cd)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.624010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.625680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 72/148] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/users/231ef980cbff45db85b8bdcef0e552cd => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:34.631633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.685509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.818074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7960ca-5e17-427f-a65d-a509ca4764b5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.867451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcccdd9-afa3-4895-a46d-dc4d1783693b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:34.875932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9652d-95cf-45c4-bbff-e260ae26bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.877011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9652d-95cf-45c4-bbff-e260ae26bb01 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.877497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9652d-95cf-45c4-bbff-e260ae26bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.877961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9652d-95cf-45c4-bbff-e260ae26bb01 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.879399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9652d-95cf-45c4-bbff-e260ae26bb01 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:34.881785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9652d-95cf-45c4-bbff-e260ae26bb01 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:34.883882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 77/149] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:34.885261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92c34-7370-4df4-b713-b975fe149b8b None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.885739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92c34-7370-4df4-b713-b975fe149b8b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.886216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92c34-7370-4df4-b713-b975fe149b8b None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.887842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 78/150] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:34.889410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92c34-7370-4df4-b713-b975fe149b8b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:34.913568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e387a8-dbf6-4b31-9adf-56c6b3a4ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:34.914214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e387a8-dbf6-4b31-9adf-56c6b3a4ad2b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.914493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e387a8-dbf6-4b31-9adf-56c6b3a4ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.914768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e387a8-dbf6-4b31-9adf-56c6b3a4ad2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.928539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e387a8-dbf6-4b31-9adf-56c6b3a4ad2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:34.932258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-11e387a8-dbf6-4b31-9adf-56c6b3a4ad2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:34.934632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:34.935474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:34.936221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 73/151] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:34.938199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:34.938486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:34.940909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:34.963380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:34.966143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:34.979454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.039683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92c34-7370-4df4-b713-b975fe149b8b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.059332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 74/152] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:35.065778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.085185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca1c9-acf5-44f5-9956-da5a6f741f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.086445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca1c9-acf5-44f5-9956-da5a6f741f82 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.086927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca1c9-acf5-44f5-9956-da5a6f741f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.087409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca1c9-acf5-44f5-9956-da5a6f741f82 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/231ef980cbff45db85b8bdcef0e552cd/roles/1c84e2fcb51c4227a0edc690959c2fb1` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.090960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ca1c9-acf5-44f5-9956-da5a6f741f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=530e3f52da9144b3b7b0c97df02943f7, user_id=231ef980cbff45db85b8bdcef0e552cd, role_id=1c84e2fcb51c4227a0edc690959c2fb1)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.092734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ca1c9-acf5-44f5-9956-da5a6f741f82 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.114794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 79/153] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:34 2022] PUT /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7/users/231ef980cbff45db85b8bdcef0e552cd/roles/1c84e2fcb51c4227a0edc690959c2fb1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:35.122545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91231698-672f-47b3-886f-ee3329fe1ac0 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.122877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91231698-672f-47b3-886f-ee3329fe1ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.123188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91231698-672f-47b3-886f-ee3329fe1ac0 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.125186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 80/154] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:35.132807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.132985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.133295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.165072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:35.166365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.192004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db726-9974-400f-a031-0ffd6ca5e520 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.265149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.265878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.266153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.267111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:35.267384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.270438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa973077-da88-40f0-95e4-f3b1a7831f26 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.291404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 75/155] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:35.292587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 81/156] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:35.295792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c6c1-fc3c-467a-878f-499f006fb27f None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.296085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c6c1-fc3c-467a-878f-499f006fb27f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.296368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495c6c1-fc3c-467a-878f-499f006fb27f None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.298145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.298532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 76/157] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:35.306180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.306456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.306736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.326729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.337109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:35.338921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:35.340850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.469317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 77/158] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:35.470456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.526555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84224f8e-791d-43c4-b182-e5c02934ec9e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.599107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.600283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.600555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.600850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.604341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.608163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.610815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 82/159] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:35.616353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.637476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265065-ba8f-43fb-9803-7c383e8f3ceb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.712734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.713465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.713747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.714017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.719877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.737714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:35.743520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:35.744453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:35.745278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:35.748878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 78/160] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:35.749927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:35.767419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:35.792480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57f8bd-af06-4af9-808c-ced7af233bd2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.841370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.842004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.842239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.842504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:35.909358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d55a2-77e4-47f9-b5fe-3e333650890e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:35.936967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:35.938454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:35.997547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:35.998121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:35.998402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:35.998672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.017094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:36.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:36.024691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:36.025459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 79/161] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:36.026950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.118155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.135699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 83/162] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:50:36.136572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.143838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.207313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53722f2-b684-4907-b8ef-d2b00f29f862 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.269514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.270147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.270329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.270600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.271383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:36.271915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.273502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.282538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 80/163] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:36.288001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.358038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e1ec6-75a6-454a-b5ba-7a629cac0146 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.444480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.445148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.445420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.445684 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.471623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:36.477890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:36.478955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:36.479701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 84/164] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:36.480343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.535481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cc717-6bcf-4628-ad79-efba1d4e5429 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.567019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.571780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.606547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.607174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.607453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.607725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.608410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:36.608785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.610235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.628492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 81/165] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:36.628914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.664879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f21344-9918-47b2-8d12-7305b2f66a49 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.719749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.720381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.720788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.721061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.722200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.723778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.731398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 85/166] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:36.732238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.822296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82ce5-a604-48d4-a318-acf59d371e78 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.884970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.885702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.885880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.886125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.889171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:36.896721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:36.899346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 82/167] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:36.905274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b47414-4cdc-4255-8558-4fd78ba8468a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:36.906950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:36.952079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:36.989745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:36.996934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:36.997465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:36.997956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:36.999616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:37.004495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.006419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.025368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 86/168] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:37.026320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.037592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.188183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f34f5-fc48-4f86-b449-2d4969b71547 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.203290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad3f09-550e-47c8-adf7-8d1faeb143ca None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.245154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.245757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.246047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.246313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.247032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=e925dc6d06704d19b20429d1596b7d2d) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:37.247332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.249062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.256410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 87/169] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/role_assignments?user.id=e925dc6d06704d19b20429d1596b7d2d&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:37.256541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.281059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.281702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.281977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.282248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] PATH_INFO: `/v3/users/00fcf728ae7844bf9da557cf68143974` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.284964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] RBAC: Authorizing `identity:get_user(user_id=00fcf728ae7844bf9da557cf68143974)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.286823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.288649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 83/170] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:36 2022] GET /identity/v3/users/00fcf728ae7844bf9da557cf68143974 => generated 247 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:37.289600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.383914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.504823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03568e48-a8a4-4a79-96e1-b3edc28a4150 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.505010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8f8a2-cfa5-4629-892f-638b8772cf2d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.530518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.565581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.566428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.566704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.567029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] PATH_INFO: `/v3/users/e925dc6d06704d19b20429d1596b7d2d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.568973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.569095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] RBAC: Authorizing `identity:get_user(user_id=e925dc6d06704d19b20429d1596b7d2d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.569897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.570177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.570682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.574120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.576985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 84/171] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/users/e925dc6d06704d19b20429d1596b7d2d => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:37.577802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.706993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.708529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.711441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672129-eb62-48ff-a3e4-3ea48313c4ce None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:37.756196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:37.757202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:37.757678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:37.758153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:37.797051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.840710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:37.862725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:37.864513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:37.926920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 88/172] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:37.928661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:37.976989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 85/173] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:37.978173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.065167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.089013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f58df6-898f-4c28-b922-33c4e91e72ac None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.153963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.154970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.155431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.155877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] PATH_INFO: `/v3/projects/dc17fc371eda4a3285526962b0406315` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.157235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] RBAC: Authorizing `identity:get_project(project_id=dc17fc371eda4a3285526962b0406315)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.159065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.160972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 86/174] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/projects/dc17fc371eda4a3285526962b0406315 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:38.162958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.174902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d062-cc47-44ab-bb82-461db0b048ab None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.269413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.270044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.270327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.270603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.300603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79fa8f-a868-4759-a0d8-d193febe73e1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.302201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.340187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.341189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.341702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.342169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.343537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.345295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.347231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 87/175] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:38 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:38.348185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.381364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.419394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.421243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.552414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ea6e3-290b-445d-a8a6-80bfb56a1491 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.605577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 89/176] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:38.606563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.645645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.646868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.647145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.647414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.649246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=dc17fc371eda4a3285526962b0406315, user.id=e925dc6d06704d19b20429d1596b7d2d) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:38.649570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.651087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.666735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 88/177] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:38 2022] GET /identity/v3/role_assignments?user.id=e925dc6d06704d19b20429d1596b7d2d&scope.project.id=dc17fc371eda4a3285526962b0406315&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:38.667604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:38.716301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.817623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7bac92-7b64-4e6e-acd5-30625f629558 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.838382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:38.857755 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee1ee-cea1-407b-b960-5c775fe81e54 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:38.859965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.860728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.861018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.861297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.862069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=00fcf728ae7844bf9da557cf68143974) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:38.862380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.863895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16e30e-0f98-4c82-9dd9-7b473ae6dcfb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.875382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 90/178] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:38 2022] GET /identity/v3/role_assignments?user.id=00fcf728ae7844bf9da557cf68143974&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:38.876846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1465598-153a-4e3d-84b1-2ddb797a1e13 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.877127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1465598-153a-4e3d-84b1-2ddb797a1e13 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.877397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1465598-153a-4e3d-84b1-2ddb797a1e13 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.878862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 91/179] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:38.880084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58660-404b-48ab-bb63-d24856c74cf9 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.880357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58660-404b-48ab-bb63-d24856c74cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.881267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58660-404b-48ab-bb63-d24856c74cf9 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.882678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 92/180] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:38.883897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5391006-182a-426c-bbab-c00cd4bb1fc5 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.884171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5391006-182a-426c-bbab-c00cd4bb1fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.884442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5391006-182a-426c-bbab-c00cd4bb1fc5 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.885854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 93/181] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:38.892394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8670-e59b-41ba-9900-1aa8ae5af204 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.892711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8670-e59b-41ba-9900-1aa8ae5af204 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.893002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d8670-e59b-41ba-9900-1aa8ae5af204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.925498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-383d8670-e59b-41ba-9900-1aa8ae5af204 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:38.925825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbab5b-8996-4c95-8196-37dd32210224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:38.926468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbab5b-8996-4c95-8196-37dd32210224 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.926719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-383d8670-e59b-41ba-9900-1aa8ae5af204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:38.926905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbab5b-8996-4c95-8196-37dd32210224 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.927063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbab5b-8996-4c95-8196-37dd32210224 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.928818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbab5b-8996-4c95-8196-37dd32210224 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:38.930346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbab5b-8996-4c95-8196-37dd32210224 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:38.932491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 89/182] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:38 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:38.933815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:38.934100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:38.934377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:38.954487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:38.956004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.008829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 94/183] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:39.014162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de38e9-b3a2-4f57-8244-6bec29c215b9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.014440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de38e9-b3a2-4f57-8244-6bec29c215b9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.014735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de38e9-b3a2-4f57-8244-6bec29c215b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.038092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 90/184] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:39.039083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.045827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-89de38e9-b3a2-4f57-8244-6bec29c215b9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:39.047044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-89de38e9-b3a2-4f57-8244-6bec29c215b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.109825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.134979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.152718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ab7c3-10e3-4476-a7b0-43e104542f1e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.160245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 95/185] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:39.161570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a574-f3e3-4587-9b04-f233a5ffc672 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.161952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a574-f3e3-4587-9b04-f233a5ffc672 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.162224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a574-f3e3-4587-9b04-f233a5ffc672 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.163498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 96/186] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:39.169662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.169978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.170207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.191910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.192646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.193789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.193942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.195012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] RBAC: Adding query filter params (role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=e925dc6d06704d19b20429d1596b7d2d) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:39.195397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.196986 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.200240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:39.201927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:39.203802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 91/187] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/role_assignments?user.id=e925dc6d06704d19b20429d1596b7d2d&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:39.281123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 97/188] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:39.440047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.442352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.559117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05bac4-a904-46b5-b2f2-c408e20c3595 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.566543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.570929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff26f3e-0407-47da-b600-c0b14df3d81e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.601732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.602696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.603184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.603669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.611385 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.612179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.616039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.616039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.616245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:39.622489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:39.623395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:39.624183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 92/189] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:39.624848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.634464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:39.638944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:39.639797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:39.640831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 98/190] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:39.641627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.751151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f928f-47ea-496a-938a-af7cff511944 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.760706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6de610-6975-403d-aa37-7091665e9fad None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.787833 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.788458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.788760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.789052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.798849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.799449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.799636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:39.799751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.799982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.800649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:39.800976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.802431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:39.805003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:39.805839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:39.806606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 93/191] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:39.807229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.813512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 99/192] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:39.813512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.838255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.874591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:39.915116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f307d2-120d-4155-bf2d-83a8e2d64a81 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.920530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40135b-b638-4d6c-8e2e-602d8c20c7c8 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:39.950497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.951125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.951404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.951678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.952896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.954362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.956489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 94/193] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:39.957200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:39.962127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:39.962735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:39.962999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:39.963269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:39.963936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:39.964224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:39.965624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:39.979735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 100/194] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:39.979735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.057717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.059008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d6450-78b6-4764-838f-a8c018d1686b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.077133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cae5f-2672-4bfa-a5f4-efaa4dc55418 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.091760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.092343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.092609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.092905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.093552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:40.093844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.095272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.096573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.103973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 95/195] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:40.104977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.113455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.114089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.114369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.114642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] PATH_INFO: `/v3/users/231ef980cbff45db85b8bdcef0e552cd` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.115900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] RBAC: Authorizing `identity:get_user(user_id=231ef980cbff45db85b8bdcef0e552cd)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.117403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.118984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 101/196] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:39 2022] GET /identity/v3/users/231ef980cbff45db85b8bdcef0e552cd => generated 289 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:40.119734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.206610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b02e8-2f2b-43d2-a948-97c57662b4fa None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.219135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c379248e-afd5-40ad-a292-2662e0bd00f6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.239361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.239981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.240262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.240645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.241330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.242788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.252038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.252653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.252955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.253221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] PATH_INFO: `/v3/users/48b2147414e149f5a84626a127bcd6bc` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.269552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] RBAC: Authorizing `identity:get_user(user_id=48b2147414e149f5a84626a127bcd6bc)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.272870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.286467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 96/197] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:40 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:40.286729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 102/198] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/users/48b2147414e149f5a84626a127bcd6bc => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:40.286824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.286906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.291834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.352440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.402968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bf86b-00b3-4a49-8fe5-f3b41e33fc23 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.415335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11994b-d66e-41e9-b7aa-8a63f1d83154 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.439138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.439753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.440029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.440296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.453422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.454080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.454358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.454884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] PATH_INFO: `/v3/users/48b2147414e149f5a84626a127bcd6bc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.456954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] RBAC: Authorizing `identity:get_user(user_id=48b2147414e149f5a84626a127bcd6bc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.458365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.461028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 97/199] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/users/48b2147414e149f5a84626a127bcd6bc => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:40.461894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.525538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.527457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.539383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.584232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c51d8-8f53-43ce-949b-483537bbae94 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.609046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.622337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.623771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.624112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.624390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.646652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 103/200] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:40.649176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.710982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.713966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.761327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedca7b6-2b4d-4910-99f9-fc254ad29268 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.772053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.800515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:40.801693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:40.802162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:40.802658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:40.837985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 98/201] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:40.838809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:40.866067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:40.881230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:40.883006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:40.964745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f5b13-517f-42a8-834a-b57255f867c8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:40.994464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 104/202] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:40.995477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.005317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.006571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.006676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.007400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.007686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.009002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.010480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.012450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 99/203] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:41.013830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.113141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.120035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3d9e5-ce7c-4c0d-971f-7a810cb75106 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.126459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccee489-d28a-48ea-8f02-99e102ec1ce9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.157907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.158545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.158821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.159091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.164162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.164162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.164162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 105/204] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:40 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:41.168042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.169239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.169352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.169772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.170183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.171463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.174275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.176096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 100/205] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:41.177326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.258807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.292525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6dad8-1f12-4941-9c40-35aac9598bee None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.303702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb76b04-f08c-4881-9adf-29b501d19652 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.329381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.330235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.330568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.330843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.331572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] RBAC: Adding query filter params (role.id=1c84e2fcb51c4227a0edc690959c2fb1, scope.project.id=530e3f52da9144b3b7b0c97df02943f7, user.id=231ef980cbff45db85b8bdcef0e552cd) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:41.331881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.333637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.345146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 106/206] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/role_assignments?user.id=231ef980cbff45db85b8bdcef0e552cd&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=1c84e2fcb51c4227a0edc690959c2fb1 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:41.345146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:41.367632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.368263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.368539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.368838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.369600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=48b2147414e149f5a84626a127bcd6bc) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:41.369904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.371526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9af2e0-a796-4136-b8a6-c4417aa45b8d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.386187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 101/207] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/role_assignments?user.id=48b2147414e149f5a84626a127bcd6bc&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:41.508006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255dcbd-36c2-4131-ac97-1679ed965cf7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:41.556388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:41.556949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:41.557227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:41.557501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:41.558249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] RBAC: Adding query filter params (role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=48b2147414e149f5a84626a127bcd6bc) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:41.558556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:41.560034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac95afd-a5a2-464e-be2c-cfc2c95752d1 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:41.572621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 107/208] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:41 2022] GET /identity/v3/role_assignments?user.id=48b2147414e149f5a84626a127bcd6bc&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:42.674035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42068cb0-dd86-4c14-9dc7-d56420bef0fa None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.674534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42068cb0-dd86-4c14-9dc7-d56420bef0fa None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.674534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42068cb0-dd86-4c14-9dc7-d56420bef0fa None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.675828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 102/209] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:42.681786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afceae-1a89-4c75-bbca-cab7ad4b0adc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.682174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afceae-1a89-4c75-bbca-cab7ad4b0adc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.682360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afceae-1a89-4c75-bbca-cab7ad4b0adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.689631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:42.702153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-45afceae-1a89-4c75-bbca-cab7ad4b0adc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:42.703324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-45afceae-1a89-4c75-bbca-cab7ad4b0adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.773628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 108/210] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:42.780870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093a32-a844-4d48-bed7-2fa74a87448d None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.781153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093a32-a844-4d48-bed7-2fa74a87448d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.781427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093a32-a844-4d48-bed7-2fa74a87448d None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.782926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 103/211] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:42.788891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:42.789172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:42.789444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:42.807722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:42.808930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:42.878340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 109/212] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:43.051014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093a32-a844-4d48-bed7-2fa74a87448d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.072946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.152232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.154453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093a32-a844-4d48-bed7-2fa74a87448d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.194141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.195001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.195442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.195852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.197365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.199022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:43.200828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 104/213] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:43.213779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.230244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.270365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6895d-1a16-415e-8384-cd5f1e92da03 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.314915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.315624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.315995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.316360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.318447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.333515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184d839-914b-450e-bbec-b2590dbd3fa4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.335268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:43.341455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:43.342338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:43.343096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:43.343821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 110/214] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:43.349801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.380122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.381286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.381564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.382029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.382690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.384132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:43.414841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 105/215] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:43 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:43.475941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b8131-ce55-4e0c-a7fa-c76526810888 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.504464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.517214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.518179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.518649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.519217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.520174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:43.520788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.522637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:43.532355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 111/216] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:43.538656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.652986 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8a5ce-c2f1-43dc-9723-21933a651feb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.689322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c674d01-d1f2-47df-ae44-f495c5d18f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.690049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674d01-d1f2-47df-ae44-f495c5d18f16 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.690228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674d01-d1f2-47df-ae44-f495c5d18f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.690491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c674d01-d1f2-47df-ae44-f495c5d18f16 None admin] PATH_INFO: `/v3/users/44f2660a00e74dd9b47127d2cc1a5f00` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.707568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c674d01-d1f2-47df-ae44-f495c5d18f16 None admin] RBAC: Authorizing `identity:get_user(user_id=44f2660a00e74dd9b47127d2cc1a5f00)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.710582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c674d01-d1f2-47df-ae44-f495c5d18f16 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:43.712747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 106/217] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/users/44f2660a00e74dd9b47127d2cc1a5f00 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:43.716817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:43.753873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4467-02dc-45bf-aaaa-64e0ed35de54 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.754171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4467-02dc-45bf-aaaa-64e0ed35de54 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.754425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4467-02dc-45bf-aaaa-64e0ed35de54 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.757054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 107/218] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:43.762278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7828c-39cc-4b0e-9211-bc5abf188529 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.762558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7828c-39cc-4b0e-9211-bc5abf188529 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.762900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7828c-39cc-4b0e-9211-bc5abf188529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.773659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.788778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-81e7828c-39cc-4b0e-9211-bc5abf188529 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:43.790746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-81e7828c-39cc-4b0e-9211-bc5abf188529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.845740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bca31-2547-47c7-80a8-a3f8ca9986df None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:43.873323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 108/219] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:43.890402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:43.891014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.891285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.891552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.923627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d17a-c058-41a0-bb4f-780d26f54ec4 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.923901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d17a-c058-41a0-bb4f-780d26f54ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.924102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d17a-c058-41a0-bb4f-780d26f54ec4 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.925743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 109/220] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:43.935126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:43.935126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:43.935126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:43.954455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:43.956058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:43.957766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:43.971928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:43.973742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.032706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 110/221] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:44.108099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 112/222] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:44.113782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.180564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.189349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:44.242924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c3b5-23c0-4596-9469-19189cb76ab4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.310892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.316698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.316698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.316698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.316698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.316698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.318202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 111/223] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:44.323710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.340722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:44.362824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ab697-7646-461f-8a58-5702a59a3975 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.414956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.415685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.416032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.416379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.430911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:44.437553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:44.438389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:44.439172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:44.440210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 113/224] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:44.440952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.445565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e1fbf-4d58-4696-9b8b-55342f20b03e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.482098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:44.492756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.493450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.493735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.494007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.509077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:44.515181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:44.517153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:44.518567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:44.524697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 112/225] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:44.524697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.578768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd923f-5606-4653-9ca1-1b446c4c60f7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.622160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.622762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.623037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.623311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.624039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=44f2660a00e74dd9b47127d2cc1a5f00) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:44.624404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.625903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.632251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 114/226] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/role_assignments?user.id=44f2660a00e74dd9b47127d2cc1a5f00&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:44.633008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.660478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee150fa-eee9-47fc-95c0-3db29720e08b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.665782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:44.706533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.707157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.707439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.707708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.708396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:44.709330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.710867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.721516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 113/227] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:44.727135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.840524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0d48e-097f-45ae-b184-cbff40430284 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.850236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:44.877785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:44.878390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:44.878710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:44.878982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:44.884698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:44.884698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:44.884698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:44.895224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 115/228] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:44.897026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:44.959741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7afd-a7cb-43b8-a5ea-88f26a3c4c66 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:44.970767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:45.004601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5565cdb-51a7-4edf-a7e4-f1ab5a90b2ee None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.040141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.040866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.041142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.041462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] PATH_INFO: `/v3/users/48b2147414e149f5a84626a127bcd6bc` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.042696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] RBAC: Authorizing `identity:get_user(user_id=48b2147414e149f5a84626a127bcd6bc)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.044160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.045777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 116/229] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/users/48b2147414e149f5a84626a127bcd6bc => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:45.051417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.065401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.066050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.066328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.066598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] PATH_INFO: `/v3/users/48b2147414e149f5a84626a127bcd6bc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.073367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] RBAC: Authorizing `identity:get_user(user_id=48b2147414e149f5a84626a127bcd6bc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.074838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.076465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 114/230] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:44 2022] GET /identity/v3/users/48b2147414e149f5a84626a127bcd6bc => generated 283 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:45.082198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.173388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c2385-4ffa-4cf5-adc3-74043e83d875 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.233432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c011e-c452-4701-b375-65232876c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.233985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c011e-c452-4701-b375-65232876c636 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.234262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c011e-c452-4701-b375-65232876c636 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.234362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bbd7-05ec-4f7d-82f0-56c290087e19 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.234538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1c011e-c452-4701-b375-65232876c636 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.235679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c011e-c452-4701-b375-65232876c636 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.237171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1c011e-c452-4701-b375-65232876c636 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.238828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 117/231] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:45 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:45.249991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c011e-c452-4701-b375-65232876c636 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.257047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:45.275434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.277010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.277295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.277575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.278463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:45.279581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.282430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.284836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 115/232] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:45 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:45.292288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:45.456236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1c011e-c452-4701-b375-65232876c636 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.476237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38be7eb-2a52-4c6a-a06e-5af031ba8a5d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:45.513998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64bda8-dd9c-4cba-ab2d-819f0d8a90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.514581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64bda8-dd9c-4cba-ab2d-819f0d8a90ef None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.514861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64bda8-dd9c-4cba-ab2d-819f0d8a90ef None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.515132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64bda8-dd9c-4cba-ab2d-819f0d8a90ef None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/48b2147414e149f5a84626a127bcd6bc/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.518271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64bda8-dd9c-4cba-ab2d-819f0d8a90ef None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc9c1d792224c5da0612bdf14e9d869, user_id=48b2147414e149f5a84626a127bcd6bc, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.519710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64bda8-dd9c-4cba-ab2d-819f0d8a90ef None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.532479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e2e13-8138-4132-a1f4-9606b9dd5c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:45.533170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e2e13-8138-4132-a1f4-9606b9dd5c8e None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:45.533497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e2e13-8138-4132-a1f4-9606b9dd5c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:45.533771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e2e13-8138-4132-a1f4-9606b9dd5c8e None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/48b2147414e149f5a84626a127bcd6bc/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:45.536168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e2e13-8138-4132-a1f4-9606b9dd5c8e None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc9c1d792224c5da0612bdf14e9d869, user_id=48b2147414e149f5a84626a127bcd6bc, role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:45.537797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e2e13-8138-4132-a1f4-9606b9dd5c8e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:45.546928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 116/233] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:45 2022] PUT /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/48b2147414e149f5a84626a127bcd6bc/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:45.579443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 118/234] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:45 2022] PUT /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/48b2147414e149f5a84626a127bcd6bc/roles/44957acdbea44978962b0cc8d2e0d069 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:46.286315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be124e8-d85f-40c4-98c8-8507b144e42e None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.287344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be124e8-d85f-40c4-98c8-8507b144e42e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.287417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be124e8-d85f-40c4-98c8-8507b144e42e None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.288867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 117/235] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:46.298552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.299087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.299087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.325689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:46.326892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.344332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b453b-c674-44d2-bd83-ddefcfcf6102 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.344642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b453b-c674-44d2-bd83-ddefcfcf6102 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.344900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b453b-c674-44d2-bd83-ddefcfcf6102 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.346234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 118/236] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:46.352049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.352344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.352625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.387472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:46.387472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:46.400020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:46.408015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 119/237] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:46.488108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:46.526453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 119/238] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:46.723177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.863785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff26c68-9e3b-4168-b4c5-4ab812014ee2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:46.910328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:46.911308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:46.912003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:46.912583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:46.913924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:46.915583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:46.918672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 120/239] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:46.925793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:46.938797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.012153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.066689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c522-28f4-48a7-aa1b-2869417b34e5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.108314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.109046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.109318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.109584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.121922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:47.126674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:47.127540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:47.128315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 121/240] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:47.135549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.198148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116eef6-2db4-4452-8a7f-d6dc8eb15e2c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.241794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b0d9-9226-4c6b-98f0-b102e2c5454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.242408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b0d9-9226-4c6b-98f0-b102e2c5454f None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.242711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b0d9-9226-4c6b-98f0-b102e2c5454f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.243194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b0d9-9226-4c6b-98f0-b102e2c5454f None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.244332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02b0d9-9226-4c6b-98f0-b102e2c5454f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.246080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02b0d9-9226-4c6b-98f0-b102e2c5454f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.254820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eeeb8-5c69-4556-a62a-80e47513971c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.273696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 120/241] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:46 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:47.275501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5d17-78c9-48f1-a1fc-f2116da502a2 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.275928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5d17-78c9-48f1-a1fc-f2116da502a2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.276376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef5d17-78c9-48f1-a1fc-f2116da502a2 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.278164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 121/242] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:47.283678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.283951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.284221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.292641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.304523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.305352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.305730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.306100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.306887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:47.307382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.308983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.310172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:47.311434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:47.319624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 122/243] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:47.325755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.356599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.417821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 122/244] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:47.516480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab465a-3c9d-42f9-85a7-b339ba089f0f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.553088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.554077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.554363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.554642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] PATH_INFO: `/v3/users/610e89fa98bc447e9853c670ff852a1f` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.569415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] RBAC: Authorizing `identity:get_user(user_id=610e89fa98bc447e9853c670ff852a1f)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.570979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.573027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 123/245] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/users/610e89fa98bc447e9853c670ff852a1f => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:47.578637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.691453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bde7b4-2dd0-4308-9107-9140d12020a7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.732365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.733018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.733357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.733640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.739234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.777169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.797086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:47.838127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:47.840388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:47.870241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1011749-2e5c-46f5-a1c5-88ce0182dbd8 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:47.915087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:47.915970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:47.916425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:47.916713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:47.932122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:47.938016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:47.938942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:47.939721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:47.942349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 124/246] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:47.948782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:47.977821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 123/247] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:47.984621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.066907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaeacd4-9d92-4898-b568-2e81538cfb13 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.076302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.094422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.123295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.123912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.124191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.124466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.125186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:48.125500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.126875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.144366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 125/248] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:48.151310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.152973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa4caf-6185-4739-9f4b-c33c73cd7509 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.220259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e801e-2dc8-4b80-9fe1-599a43249fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.220933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e801e-2dc8-4b80-9fe1-599a43249fcb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.221213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e801e-2dc8-4b80-9fe1-599a43249fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.221486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e801e-2dc8-4b80-9fe1-599a43249fcb None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.225576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e801e-2dc8-4b80-9fe1-599a43249fcb None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.227074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e801e-2dc8-4b80-9fe1-599a43249fcb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.231109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 124/249] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:47 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:48.233486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119581a6-f49c-4c6b-bca6-940c9d12b72f None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.233773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119581a6-f49c-4c6b-bca6-940c9d12b72f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.234051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119581a6-f49c-4c6b-bca6-940c9d12b72f None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.235390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 125/250] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:48.238527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119581a6-f49c-4c6b-bca6-940c9d12b72f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.362414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119581a6-f49c-4c6b-bca6-940c9d12b72f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.389055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050da29-a557-4663-b288-c5286b581e3f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.400806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.407657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.408557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.409050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.409507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.410979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] RBAC: Adding query filter params (role.id=1c84e2fcb51c4227a0edc690959c2fb1, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=610e89fa98bc447e9853c670ff852a1f) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:48.411647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.414192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52af9cd-51ea-4820-96fc-947f007d2efa None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.421155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 126/251] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/role_assignments?user.id=610e89fa98bc447e9853c670ff852a1f&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=1c84e2fcb51c4227a0edc690959c2fb1 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:48.422922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b4b0b-b85f-4d3b-bb64-645d86766588 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.423732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b4b0b-b85f-4d3b-bb64-645d86766588 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.424220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b4b0b-b85f-4d3b-bb64-645d86766588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.446841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-152b4b0b-b85f-4d3b-bb64-645d86766588 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:48.449362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-152b4b0b-b85f-4d3b-bb64-645d86766588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.507153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.507825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.508109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.508386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] PATH_INFO: `/v3/users/44f2660a00e74dd9b47127d2cc1a5f00` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.513508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=44f2660a00e74dd9b47127d2cc1a5f00)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.515013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.517125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 126/252] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/users/44f2660a00e74dd9b47127d2cc1a5f00 => generated 297 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:48.539608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.541347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 127/253] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:48.605452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.700836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc2ceb-ca9b-47e0-bb5e-a77e27046ba2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.721517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458ed7-476f-4d13-9ad6-326f23a3ec55 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.722866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458ed7-476f-4d13-9ad6-326f23a3ec55 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.723339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4458ed7-476f-4d13-9ad6-326f23a3ec55 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.726293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 128/254] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:48.735041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.735914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.736386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.769737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:48.771802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:48.775639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:48.775789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:48.776003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:48.776223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:48.781420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:48.783195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:48.784959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 127/255] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:48.792203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:48.883725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 129/256] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:48.954899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ec29a-c190-4224-83eb-57d61748f56e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:48.957243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:48.995115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.004960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.005670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.005952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.006241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/44f2660a00e74dd9b47127d2cc1a5f00/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.013433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc9c1d792224c5da0612bdf14e9d869, user_id=44f2660a00e74dd9b47127d2cc1a5f00, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.014907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.039932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 128/257] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:48 2022] PUT /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/44f2660a00e74dd9b47127d2cc1a5f00/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:50:49.084658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.144342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864ab5b-85ed-4b8c-875c-c62d7ce468e0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.185674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.186321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.186597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.186871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.201941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:49.207502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:49.208461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:50:49.209721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:49.212820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 130/258] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:50:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:49.215438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.358956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a3615-033e-478e-b666-1d04f24f3b98 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.417433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.418089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.418332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.418603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.419289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:49.419590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.421033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.431351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 129/259] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:49.437105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.452939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.513119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.562026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ccb15-ea5e-4746-a7a5-296699942b4f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.582228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.605117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.605748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.606024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.606294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] PATH_INFO: `/v3/users/48b2147414e149f5a84626a127bcd6bc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.607648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] RBAC: Authorizing `identity:get_user(user_id=48b2147414e149f5a84626a127bcd6bc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.609128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:49.610918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 131/260] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/users/48b2147414e149f5a84626a127bcd6bc => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:49.617416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.669128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a70f20-5e8e-4459-8c54-7fedd464eff8 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.712894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.713542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.713830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.714107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.735198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:49.745605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:49.746498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:49.747915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 130/261] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:49.752340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:49.776786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7c378-ac84-4a78-a637-4c453aa0d36d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.864042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:49.865617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:49.866174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:49.866646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:49.902459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.950966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acd191-0b7e-45d0-a645-8c39a0a20270 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:49.968701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:49.984273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:49.987838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.058202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.058708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.058995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.059275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.059956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:50.060258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.062340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.082784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 131/262] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:50.093801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.098489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 132/263] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:50:50.110435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.257145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.273199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da26ecc-870b-41ec-a9e2-f667e89bdd4f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.292023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93194c-239f-4768-a238-567f9e3193e7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.328494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.329094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.329459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.332710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] PATH_INFO: `/v3/users/48b2147414e149f5a84626a127bcd6bc` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.332710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] RBAC: Authorizing `identity:get_user(user_id=48b2147414e149f5a84626a127bcd6bc)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.332710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.334780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 132/264] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/users/48b2147414e149f5a84626a127bcd6bc => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:50.339956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.345358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.346230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.346650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.347063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.353536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.355282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.357350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 133/265] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:50.369431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.371105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.554778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b11a79-588c-44a6-89f8-4519f5fd4123 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.609928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643833de-1e6b-476f-84a3-975d8e83c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.610462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643833de-1e6b-476f-84a3-975d8e83c962 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.610741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643833de-1e6b-476f-84a3-975d8e83c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.611012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643833de-1e6b-476f-84a3-975d8e83c962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.623130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d8641-68fb-4e3d-bcb9-6fbcf8354b2f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:50.690829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643833de-1e6b-476f-84a3-975d8e83c962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.692536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643833de-1e6b-476f-84a3-975d8e83c962 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.725096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.781521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:50.782152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.782432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.782944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.783883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] RBAC: Adding query filter params (role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=48b2147414e149f5a84626a127bcd6bc) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:50.784391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:50.786167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:50.799963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 133/266] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:50.801619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d274b-9efa-4383-af27-8c01b9593924 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.802721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d274b-9efa-4383-af27-8c01b9593924 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.803193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d274b-9efa-4383-af27-8c01b9593924 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.803988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 134/267] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/role_assignments?user.id=48b2147414e149f5a84626a127bcd6bc&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 608 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:50.804920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 134/268] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:50.808550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:50.818604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:50.819197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:50.819617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:50.850499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:50.852299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:50.870807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:50.962828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 135/269] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:51.065967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aee514-03a1-41d3-abf9-c7929f1f6277 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.110057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d4e8ee-cadd-4489-b7a0-573c7370593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.110704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4e8ee-cadd-4489-b7a0-573c7370593e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.110986 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4e8ee-cadd-4489-b7a0-573c7370593e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.111265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d4e8ee-cadd-4489-b7a0-573c7370593e None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.112416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4e8ee-cadd-4489-b7a0-573c7370593e None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.113896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d4e8ee-cadd-4489-b7a0-573c7370593e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.116160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 135/270] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:50 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:51.122079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.208088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.277638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974dbcc-b00e-49ad-8001-f19277135a36 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.306026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01073a3b-2563-47d3-a802-b5522b5c694c None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.306564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01073a3b-2563-47d3-a802-b5522b5c694c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.306983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01073a3b-2563-47d3-a802-b5522b5c694c None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.308872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 136/271] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:51.322001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.322608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.322885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.323158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.323890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=48b2147414e149f5a84626a127bcd6bc) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:51.324191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.325672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.336351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 136/272] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/role_assignments?user.id=48b2147414e149f5a84626a127bcd6bc&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:51.338101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.338598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.339073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.399011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:51.400633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.424150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.503801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 137/273] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:51.623072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c510203-b712-4940-aa27-a5fceaa63fd5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.637904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.674387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b974acd-f8a8-4183-8dcb-822379ef8dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.674933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b974acd-f8a8-4183-8dcb-822379ef8dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.675205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b974acd-f8a8-4183-8dcb-822379ef8dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.675471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b974acd-f8a8-4183-8dcb-822379ef8dd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.692232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b974acd-f8a8-4183-8dcb-822379ef8dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-2b974acd-f8a8-4183-8dcb-822379ef8dd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:51.698447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:51.699369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:51.700873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 137/274] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:51.706049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.735061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.828384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581ba0e-9763-44a9-8e1b-247bbe371bf2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.829549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b33e-eb89-4753-862f-590a3de8e1da None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.830561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b33e-eb89-4753-862f-590a3de8e1da None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.830742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b33e-eb89-4753-862f-590a3de8e1da None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.832231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 138/275] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:51.833010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b33e-eb89-4753-862f-590a3de8e1da None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.865505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.866115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.866399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.866677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.867339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:51.867640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.869042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb59f42-630e-45fc-89a7-89333924feb0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.877471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 138/276] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:51.884031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.884323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.884605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.903546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:51.904758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:51.941806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b33e-eb89-4753-862f-590a3de8e1da None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:51.976116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:51.980798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:51.980925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 139/277] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:51.981002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:51.981002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:51.981165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:51.981503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:51.983942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:51.990433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 139/278] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:51.991475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:51.992547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.042278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.093501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73a67e-e2e6-4554-aa96-5b1e267f901b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.112312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4fdd2-fc87-4d78-8468-8f7472147313 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.130438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7bfb2-89a9-43c9-859d-f8e9cd05fc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.131120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bfb2-89a9-43c9-859d-f8e9cd05fc98 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.131401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bfb2-89a9-43c9-859d-f8e9cd05fc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.131679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7bfb2-89a9-43c9-859d-f8e9cd05fc98 None admin] PATH_INFO: `/v3/users/1f3a10603ef04554be5f7cff0def8048` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.146005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7bfb2-89a9-43c9-859d-f8e9cd05fc98 None admin] RBAC: Authorizing `identity:get_user(user_id=1f3a10603ef04554be5f7cff0def8048)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.147470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7bfb2-89a9-43c9-859d-f8e9cd05fc98 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.149177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 140/279] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:51 2022] GET /identity/v3/users/1f3a10603ef04554be5f7cff0def8048 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:52.150453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacf256-5868-4716-af10-83e8ac4e4937 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.150738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacf256-5868-4716-af10-83e8ac4e4937 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.151012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacf256-5868-4716-af10-83e8ac4e4937 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.152340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 141/280] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:52.154801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacf256-5868-4716-af10-83e8ac4e4937 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.158404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0289fe09-e7ba-4abd-bc81-3116ea3ae95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.159955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289fe09-e7ba-4abd-bc81-3116ea3ae95c None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.160234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289fe09-e7ba-4abd-bc81-3116ea3ae95c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.160505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0289fe09-e7ba-4abd-bc81-3116ea3ae95c None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.161180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0289fe09-e7ba-4abd-bc81-3116ea3ae95c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.163188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0289fe09-e7ba-4abd-bc81-3116ea3ae95c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.186429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 140/281] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:50:51 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:52.187374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.187661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.187937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.208789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:52.210005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:52.241363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.289089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacf256-5868-4716-af10-83e8ac4e4937 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.296719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 141/282] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:52.296719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.324396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.344414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.345324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.345739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.346147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.347695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, user.id=1f3a10603ef04554be5f7cff0def8048, scope.system=all) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:52.348153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.350795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.358079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 142/283] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/role_assignments?user.id=1f3a10603ef04554be5f7cff0def8048&scope.system=all&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:52.432479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68d5ff-9445-487a-a738-211eda40e5b3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.472173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.472759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.473040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.473311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.486322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:52.492290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:52.493458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:52.494329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 142/284] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:52.499744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.571091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.612059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3febe1-43c8-46bc-8924-ade0c3eac7bb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.641242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.649580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.650299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.650657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.650940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.651621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:52.652732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.653368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.667364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 143/285] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:52.669787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.680583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.700944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82573b-2d25-4f0b-a476-446607c7ec4e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.747033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f20b4-4926-402f-827b-820f35245307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.748286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f20b4-4926-402f-827b-820f35245307 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.748937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f20b4-4926-402f-827b-820f35245307 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.749411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f20b4-4926-402f-827b-820f35245307 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.767503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f20b4-4926-402f-827b-820f35245307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-220f20b4-4926-402f-827b-820f35245307 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:52.782013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:52.782924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:52.783695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:52.789430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 143/286] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:52.796235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f20b4-4926-402f-827b-820f35245307 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.835219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142c1b3-6997-4bda-b6b1-e50482d5e5ba None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:52.888111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:52.888778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:52.889060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:52.889332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] PATH_INFO: `/v3/users/610e89fa98bc447e9853c670ff852a1f` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:52.890473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] RBAC: Authorizing `identity:get_user(user_id=610e89fa98bc447e9853c670ff852a1f)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:52.891914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:52.893643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 144/287] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/users/610e89fa98bc447e9853c670ff852a1f => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:52.900077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:52.971861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:52.994744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f20b4-4926-402f-827b-820f35245307 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.039566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.040217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.040496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.040799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.041468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:53.041763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.043322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.051163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9cb4a-c174-4144-bd6a-289ca56ce61e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.056710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 144/288] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:53.059489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.077016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.121311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.121944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.122219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.122488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.123714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.126390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.129407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 145/289] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:52 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:53.135598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.230097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d2736-3f01-4828-b539-cba63b5855a2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.294039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.295124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.295721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.296195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] PATH_INFO: `/v3/users/44f2660a00e74dd9b47127d2cc1a5f00` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.297711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=44f2660a00e74dd9b47127d2cc1a5f00)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.298532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c53950-41ad-477c-9a1c-3f3212402c2d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.308077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.310608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 145/290] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/users/44f2660a00e74dd9b47127d2cc1a5f00 => generated 297 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:53.316469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.333691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.343303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.343958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.344235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.344506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/610e89fa98bc447e9853c670ff852a1f/roles/1c84e2fcb51c4227a0edc690959c2fb1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.350765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc9c1d792224c5da0612bdf14e9d869, user_id=610e89fa98bc447e9853c670ff852a1f, role_id=1c84e2fcb51c4227a0edc690959c2fb1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.355710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.385919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 146/291] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:50:53 2022] PUT /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/users/610e89fa98bc447e9853c670ff852a1f/roles/1c84e2fcb51c4227a0edc690959c2fb1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:53.475337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.517119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f86b13-4e05-48d0-a1a8-7e407d21f6c3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.568253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.568810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.569082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.569348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.651373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.652895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.742245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:53.779233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 146/292] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:50:53.785061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:53.922047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee4ca7-5f28-4631-90d0-e1daca6b3626 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:53.958065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01baacc-166b-4ebb-9d80-822fe6595673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:53.958727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01baacc-166b-4ebb-9d80-822fe6595673 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:53.959007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01baacc-166b-4ebb-9d80-822fe6595673 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:53.959283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01baacc-166b-4ebb-9d80-822fe6595673 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:53.960411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01baacc-166b-4ebb-9d80-822fe6595673 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:53.961906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01baacc-166b-4ebb-9d80-822fe6595673 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:53.963767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 147/293] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:53.969472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:54.085778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256f890-ae94-4d4d-b6d8-0d8109fc3281 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:54.131351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:54.131956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.132230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.132506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.133270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=44f2660a00e74dd9b47127d2cc1a5f00) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:54.133574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:54.135012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5819f87-f5d2-4a6b-8b0e-b74a8159d6cd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:54.141473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 147/294] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:53 2022] GET /identity/v3/role_assignments?user.id=44f2660a00e74dd9b47127d2cc1a5f00&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:54.806557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bd12e-4a3c-4480-9dd9-9306235130eb None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.806972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bd12e-4a3c-4480-9dd9-9306235130eb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.807147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bd12e-4a3c-4480-9dd9-9306235130eb None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.808810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 148/295] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:54.814221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb269b50-0f22-4870-9dcb-f2df456dbef7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:54.814677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb269b50-0f22-4870-9dcb-f2df456dbef7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:54.814753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb269b50-0f22-4870-9dcb-f2df456dbef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:54.836885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-cb269b50-0f22-4870-9dcb-f2df456dbef7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:54.838139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-cb269b50-0f22-4870-9dcb-f2df456dbef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:54.874924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:54.922448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 148/296] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:55.074114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c82c-9d2c-4400-9325-f93827988d8c None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.074608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c82c-9d2c-4400-9325-f93827988d8c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.074608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436c82c-9d2c-4400-9325-f93827988d8c None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.076278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 149/297] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:55.082906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864765-2273-4fad-8d5c-7bc2a2d2f4b8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.084069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864765-2273-4fad-8d5c-7bc2a2d2f4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.084780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28864765-2273-4fad-8d5c-7bc2a2d2f4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.106075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-28864765-2273-4fad-8d5c-7bc2a2d2f4b8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:55.107566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-28864765-2273-4fad-8d5c-7bc2a2d2f4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.176852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 149/298] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:55.186248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b742-2afc-4424-b3d4-813ada6ff2d0 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.186809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b742-2afc-4424-b3d4-813ada6ff2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.187278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174b742-2afc-4424-b3d4-813ada6ff2d0 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.189066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 150/299] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:55.194878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.195154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.195422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.214786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:55.215943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:55.217623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174b742-2afc-4424-b3d4-813ada6ff2d0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.280585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.305294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 150/300] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:55.342649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174b742-2afc-4424-b3d4-813ada6ff2d0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.383852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.384737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.385151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.385588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.402396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:55.407902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:55.408997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:55.409956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:55.411817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 151/301] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:55.417738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.446943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.463205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.523450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0ea00-7ac4-47ed-bbc8-33e387607a91 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.559895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.560869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.561338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.561797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.562669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:55.563165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.564752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.565331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79880065-7b19-4d43-adcf-b356071a5c97 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.574304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 151/302] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:55.580221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.580455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.602090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.602746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.603021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.603286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.615088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:55.619961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:55.620837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:55.621604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 152/303] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:55.625601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.686467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.697334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6466d23-d945-4caf-8783-5577a10e5a71 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.733713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.734355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.734629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.734899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] PATH_INFO: `/v3/users/1f3a10603ef04554be5f7cff0def8048` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.736101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] RBAC: Authorizing `identity:get_user(user_id=1f3a10603ef04554be5f7cff0def8048)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.736211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c42fa-ad08-45e4-b259-ceb425b93823 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.737638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.740343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 152/304] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/users/1f3a10603ef04554be5f7cff0def8048 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:55.745746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.777673 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.778308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.778581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.778855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.779941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.781413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.783271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 153/305] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:55.784011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.833710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.856483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c3240-fac9-4d4a-968d-c1634767c7cd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.892141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.892349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c265a-a542-4f6c-af6d-3c42908adfe5 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:55.892770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.893052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.893328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.893991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:55.894298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.895709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.903826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 153/306] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:55.909454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:55.925509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01726e-cae5-4d35-9976-79131f22c4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:55.926157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01726e-cae5-4d35-9976-79131f22c4b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.926433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01726e-cae5-4d35-9976-79131f22c4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.926706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01726e-cae5-4d35-9976-79131f22c4b4 None admin] PATH_INFO: `/v3/system/users/1f3a10603ef04554be5f7cff0def8048/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.928778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01726e-cae5-4d35-9976-79131f22c4b4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f3a10603ef04554be5f7cff0def8048, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:55.930257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01726e-cae5-4d35-9976-79131f22c4b4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:55.932861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:55.939796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 154/307] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Sep 17 20:50:55 2022] PUT /identity/v3/system/users/1f3a10603ef04554be5f7cff0def8048/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:50:55.941122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4158dc-91dd-40e3-8d49-eb8c77d9b9ff None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:55.941445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4158dc-91dd-40e3-8d49-eb8c77d9b9ff None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:55.941834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4158dc-91dd-40e3-8d49-eb8c77d9b9ff None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:55.943817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 155/308] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:55.944602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4158dc-91dd-40e3-8d49-eb8c77d9b9ff None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.026063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f588a97-725d-48d7-b819-c340c4838576 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.062336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8d067-0416-439b-861b-9eb326798692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.062949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d067-0416-439b-861b-9eb326798692 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.063289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d067-0416-439b-861b-9eb326798692 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.063563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8d067-0416-439b-861b-9eb326798692 None admin] PATH_INFO: `/v3/groups` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.064205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8d067-0416-439b-861b-9eb326798692 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.065471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4158dc-91dd-40e3-8d49-eb8c77d9b9ff None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.065735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8d067-0416-439b-861b-9eb326798692 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.073916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.082880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 154/309] 10.222.0.16 () {60 vars in 1134 bytes} [Sat Sep 17 20:50:55 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:56.091153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.091473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.091701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.116189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.116791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.117073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.117403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] PATH_INFO: `/v3/users/4b5871589c96468182a85b46c20aa5cb` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.129053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:56.130362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:56.135174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] RBAC: Authorizing `identity:get_user(user_id=4b5871589c96468182a85b46c20aa5cb)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.137033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.139613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 156/310] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:55 2022] GET /identity/v3/users/4b5871589c96468182a85b46c20aa5cb => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:56.146837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.234849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 155/311] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:56.265709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d084b12-cd48-4c35-930f-4714279f235e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.303249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.303851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.304129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.304400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.305162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] RBAC: Adding query filter params (role.id=1c84e2fcb51c4227a0edc690959c2fb1, user.id=4b5871589c96468182a85b46c20aa5cb, scope.system=all) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:56.305465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.306899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.312552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 157/312] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 17 20:50:56 2022] GET /identity/v3/role_assignments?user.id=4b5871589c96468182a85b46c20aa5cb&scope.system=all&role.id=1c84e2fcb51c4227a0edc690959c2fb1 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:56.510490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.524007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.618925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb5cbb-514d-450a-9b5e-0146122ae389 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.654760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.655315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.655590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.655861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.666359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:56.671174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:56.672054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:50:56.673034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 156/313] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:56.682097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.712687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.789494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387e0e6-9dbc-4177-ac7f-9740f2d0fa73 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.825624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.826508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.826932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.827348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.828160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:56.828602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.830415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:56.839765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 158/314] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:56.845718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:56.923856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:56.960589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f0a9-5303-4ccc-8ed9-7a207f66e1e6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:56.995241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:56.995861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:56.996130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:56.996396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] PATH_INFO: `/v3/users/610e89fa98bc447e9853c670ff852a1f` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:56.997540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] RBAC: Authorizing `identity:get_user(user_id=610e89fa98bc447e9853c670ff852a1f)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:56.998971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.000847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 157/315] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:56 2022] GET /identity/v3/users/610e89fa98bc447e9853c670ff852a1f => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:57.009783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.113230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:57.123697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490bc54-e1dc-4391-bae3-c06a316d3cf6 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.159263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.160169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.160626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.161099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.231462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.233239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.339571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:57.351012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 159/316] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:50:57.356550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.468952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7232a5f-ed31-4c01-9f27-17a967ae5409 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.503466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c3ed-9377-49df-a4f7-6e566786b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.504355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c3ed-9377-49df-a4f7-6e566786b8fd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.504829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c3ed-9377-49df-a4f7-6e566786b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.505858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c3ed-9377-49df-a4f7-6e566786b8fd None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.507162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35c3ed-9377-49df-a4f7-6e566786b8fd None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.508805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35c3ed-9377-49df-a4f7-6e566786b8fd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.510892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 158/317] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:57.516703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:57.616730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2383b-8814-4ca2-b8d5-1ab9ed28433d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:57.653444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:57.654048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:57.654326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:57.654599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:57.655328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] RBAC: Adding query filter params (role.id=1c84e2fcb51c4227a0edc690959c2fb1, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869, user.id=610e89fa98bc447e9853c670ff852a1f) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:57.655634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:57.657106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd9e41-8a0c-4a31-887e-8d7d9658db5d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:57.663499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 160/318] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:50:57 2022] GET /identity/v3/role_assignments?user.id=610e89fa98bc447e9853c670ff852a1f&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=1c84e2fcb51c4227a0edc690959c2fb1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:58.275270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81782b9d-6ea7-4e98-962f-052ca67a56ea None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.275775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81782b9d-6ea7-4e98-962f-052ca67a56ea None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.275775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81782b9d-6ea7-4e98-962f-052ca67a56ea None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.277176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 159/319] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:58.282473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f71c-b322-448c-a235-3475a3e9ac1e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.282857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f71c-b322-448c-a235-3475a3e9ac1e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.283000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f71c-b322-448c-a235-3475a3e9ac1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.301818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5590f71c-b322-448c-a235-3475a3e9ac1e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:58.303601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5590f71c-b322-448c-a235-3475a3e9ac1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.354373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.356947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42730a-afd2-409a-8a49-60c380ff5318 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.357243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42730a-afd2-409a-8a49-60c380ff5318 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.357553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42730a-afd2-409a-8a49-60c380ff5318 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.358837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 160/320] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:58.363891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6223176-decf-4f97-be81-eeafbfe0373a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.364955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6223176-decf-4f97-be81-eeafbfe0373a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.365389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6223176-decf-4f97-be81-eeafbfe0373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.375756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 161/321] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:58.383670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.385547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b6223176-decf-4f97-be81-eeafbfe0373a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:58.389934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b6223176-decf-4f97-be81-eeafbfe0373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.458934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 161/322] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:50:58.584342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95edda0-55de-493d-b3fb-19dca4a391f9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.584533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95edda0-55de-493d-b3fb-19dca4a391f9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.585022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95edda0-55de-493d-b3fb-19dca4a391f9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.586519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 162/323] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:50:58.591429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.591712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.591982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.610317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:50:58.611495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:50:58.653245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95edda0-55de-493d-b3fb-19dca4a391f9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.679932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 162/324] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:50:58.723036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.758653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95edda0-55de-493d-b3fb-19dca4a391f9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.792963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.797470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.798095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.798376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.798648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.808939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:58.814317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:58.815378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:58.817702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 163/325] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:50:58.821531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.837297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f11e76-0f74-4f56-a9d9-bac28cf13148 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.869381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:58.874367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.874990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.875268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.875538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.876626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.878098 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.884815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 163/326] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:50:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:58.886359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.929777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ea8cf-8d44-4cec-8a3b-6768a140fbed None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:58.963419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:58.964021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:58.964293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:58.964561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:58.965244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:58.965546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:58.966897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:58.975270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 164/327] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:58.976050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:58.988836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e174fe0-145f-41b6-aba5-e520185dae81 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.022756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.023349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.023621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.023889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] PATH_INFO: `/v3/groups` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.024512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.025975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.029818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.042963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 164/328] 10.222.0.16 () {60 vars in 1134 bytes} [Sat Sep 17 20:50:58 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:50:59.043770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.086305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b43f2-81c4-403b-ac73-8632bd2ed78f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.094218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.128842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.129500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.129803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.130129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.142435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:50:59.147446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:50:59.148342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:50:59.149340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 165/329] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:50:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:50:59.154552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.161845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24810bf8-9739-42e4-9cfd-106373274363 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.197985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.198629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.198905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.199177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] PATH_INFO: `/v3/users/1f3a10603ef04554be5f7cff0def8048` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.200352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] RBAC: Authorizing `identity:get_user(user_id=1f3a10603ef04554be5f7cff0def8048)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.201878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.203513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 165/330] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/users/1f3a10603ef04554be5f7cff0def8048 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:50:59.209935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.272474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3244d22-dd1e-4d64-8806-814b38ae7653 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.282104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.317149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.317980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.318263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.318534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.319209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:59.319510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.320948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.329969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 166/331] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:59.330380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78986d0-1104-439c-bf1b-11fa6728f6dc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.336234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.369201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.371940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.372556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.372863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.373133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.373867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] RBAC: Adding query filter params (role.id=e5bac5bfcc784719ac8f886a28cb9df6, user.id=1f3a10603ef04554be5f7cff0def8048, scope.system=all) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:50:59.374171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.375613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.390757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 166/332] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/role_assignments?user.id=1f3a10603ef04554be5f7cff0def8048&scope.system=all&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:59.473224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f3ef3-4936-4115-b44a-a19e0c328387 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.515838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dc126-c208-40e2-abe1-db86bd5598f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.516487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc126-c208-40e2-abe1-db86bd5598f1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.516792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc126-c208-40e2-abe1-db86bd5598f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.517069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dc126-c208-40e2-abe1-db86bd5598f1 None admin] PATH_INFO: `/v3/users/4b5871589c96468182a85b46c20aa5cb` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.518267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dc126-c208-40e2-abe1-db86bd5598f1 None admin] RBAC: Authorizing `identity:get_user(user_id=4b5871589c96468182a85b46c20aa5cb)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.519725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dc126-c208-40e2-abe1-db86bd5598f1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.522259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 167/333] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:50:59 2022] GET /identity/v3/users/4b5871589c96468182a85b46c20aa5cb => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:50:59.527690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:50:59.641421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74862-3e0f-470e-a4db-9d9985b3568b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:50:59.664283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:59.680339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12b43f-7293-4998-bf74-64758bf5d9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:50:59.681038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b43f-7293-4998-bf74-64758bf5d9c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:50:59.681316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b43f-7293-4998-bf74-64758bf5d9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:50:59.681597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b43f-7293-4998-bf74-64758bf5d9c9 None admin] PATH_INFO: `/v3/system/users/4b5871589c96468182a85b46c20aa5cb/roles/1c84e2fcb51c4227a0edc690959c2fb1` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:50:59.683049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12b43f-7293-4998-bf74-64758bf5d9c9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4b5871589c96468182a85b46c20aa5cb, role_id=1c84e2fcb51c4227a0edc690959c2fb1)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:50:59.684443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12b43f-7293-4998-bf74-64758bf5d9c9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:50:59.696359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 167/334] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Sep 17 20:50:59 2022] PUT /identity/v3/system/users/4b5871589c96468182a85b46c20aa5cb/roles/1c84e2fcb51c4227a0edc690959c2fb1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:02.065471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a9dd-410d-48e6-b86e-7037f8b2eee9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.066000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a9dd-410d-48e6-b86e-7037f8b2eee9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.066000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882a9dd-410d-48e6-b86e-7037f8b2eee9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.068203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 168/335] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:02.086240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228868f-d701-450f-a65e-f2c5562dbd94 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.086809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228868f-d701-450f-a65e-f2c5562dbd94 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.086809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228868f-d701-450f-a65e-f2c5562dbd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.113739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a228868f-d701-450f-a65e-f2c5562dbd94 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.115374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a228868f-d701-450f-a65e-f2c5562dbd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.201633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 168/336] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:02.244943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549141d-6eb6-4db4-9366-027eca28d74d None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.245222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549141d-6eb6-4db4-9366-027eca28d74d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.245640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549141d-6eb6-4db4-9366-027eca28d74d None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.247049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 169/337] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:02.252430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee71dc-fd0f-48d1-83eb-ba12ce085426 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.252733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee71dc-fd0f-48d1-83eb-ba12ce085426 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.253015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee71dc-fd0f-48d1-83eb-ba12ce085426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.275290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e6ee71dc-fd0f-48d1-83eb-ba12ce085426 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.276477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e6ee71dc-fd0f-48d1-83eb-ba12ce085426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.361357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 169/338] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:02.509656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2648-c45a-44a4-aacf-de2fc233b9a9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.510022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2648-c45a-44a4-aacf-de2fc233b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.510194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a2648-c45a-44a4-aacf-de2fc233b9a9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.511794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 170/339] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:02.516872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.517159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.517438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.545804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:02.547018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:02.597932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:02.633225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 170/340] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:02.678462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2648-c45a-44a4-aacf-de2fc233b9a9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.787510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:02.796473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a2648-c45a-44a4-aacf-de2fc233b9a9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:02.833652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:02.853194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:02.853832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:02.854111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:02.854381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:02.881475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:02.888264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:02.889217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:02.890023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 171/341] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:02.898559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.025503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ba2c-b8c4-4427-a440-4a9b531579dc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.088835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab24eab-d72e-42e2-abc3-d3d95c0936bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.089666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab24eab-d72e-42e2-abc3-d3d95c0936bb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.089830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab24eab-d72e-42e2-abc3-d3d95c0936bb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.090086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab24eab-d72e-42e2-abc3-d3d95c0936bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.094746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d08478-2c19-4b59-8815-5e51c084cadd None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.108179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab24eab-d72e-42e2-abc3-d3d95c0936bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-4ab24eab-d72e-42e2-abc3-d3d95c0936bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:03.113080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:03.113938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:03.114821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 171/342] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:03.116167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cc07d-5136-4562-a7b5-5067743839e5 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.116447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cc07d-5136-4562-a7b5-5067743839e5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.119778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cc07d-5136-4562-a7b5-5067743839e5 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.121353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 172/343] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:03.122553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1866-9bfd-49fe-b6c1-b7c0b11a03ab None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.125323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1866-9bfd-49fe-b6c1-b7c0b11a03ab None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.125323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e1866-9bfd-49fe-b6c1-b7c0b11a03ab None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.127034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 173/344] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:03.127034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e1866-9bfd-49fe-b6c1-b7c0b11a03ab None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.137750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.138417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.138644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.138950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.139621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:03.139952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.141359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.151794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 172/345] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:03.152714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.177229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.177884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.263668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e1866-9bfd-49fe-b6c1-b7c0b11a03ab None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.291014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a3027-6336-45b6-a364-1a8ebac5ca72 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.304866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce81ea-54e0-4eac-9178-181291f54347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.305855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce81ea-54e0-4eac-9178-181291f54347 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.306461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce81ea-54e0-4eac-9178-181291f54347 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.307328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce81ea-54e0-4eac-9178-181291f54347 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.320285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce81ea-54e0-4eac-9178-181291f54347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-60ce81ea-54e0-4eac-9178-181291f54347 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:03.325519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:03.326399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:03.327257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 174/346] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:03.328430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.328430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.328604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.335142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.335730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.336010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.336661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.336975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:03.337279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.339764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446089dd-bc28-4fb9-b8ca-9051ae359aa3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.350685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 173/347] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:03.351993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:03.352210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.352470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.352776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.354056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:03.378642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:03.381006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:03.440421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 175/348] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:03.441529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.469926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 174/349] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:03.473637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.485438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.498419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.564578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cf33f-2669-4cc3-9c4f-00a69d79f9ca None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.630901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.631514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.631786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.632054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.657416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d34f44-6003-4daf-b609-61ff2dbb20a5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.716091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.716849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.717127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.717396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.718071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:03.718450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.719939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.737733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 175/350] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:03.738426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.743420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:03.745029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:03.785945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.860724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:03.885731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 176/351] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:03.886814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:03.895925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e0f09-338e-4912-9c96-d8c6ae28b331 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:03.936235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:03.937123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:03.937123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:03.937346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:03.999422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340f27d-fb4b-46cd-9a68-347d4cafd896 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.019349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.021866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.028409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.042358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.042985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.043261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.043534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.116927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.153129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.155273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.159668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 176/352] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:04.166629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.268475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 177/353] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:04.269521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.284380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc3da3-5246-4f2a-a465-37341eb92be4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.296427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.330208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.331172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.331645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.331920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.342336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:04.347240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:04.348108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:04.349726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 177/354] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:04.354488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.389547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.397499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54839563-11c9-414b-a14c-b55adabca2c5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.438442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.439066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.439346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.439615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.452567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:04.458093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:04.459422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:04.460207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 178/355] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:04.467554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.482327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8ef83-aec4-483e-87ad-687e8883f7ef None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.525501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e9738-1a4b-47f1-a65a-0010ab8afb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.526624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e9738-1a4b-47f1-a65a-0010ab8afb16 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.526914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e9738-1a4b-47f1-a65a-0010ab8afb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.527190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e9738-1a4b-47f1-a65a-0010ab8afb16 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.528640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e9738-1a4b-47f1-a65a-0010ab8afb16 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.530141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e9738-1a4b-47f1-a65a-0010ab8afb16 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.532417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 178/356] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:04.533596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51af3-297c-4852-a6ae-e9aacdf6de30 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.533878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51af3-297c-4852-a6ae-e9aacdf6de30 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.534152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51af3-297c-4852-a6ae-e9aacdf6de30 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.535428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 179/357] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:04.541086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51af3-297c-4852-a6ae-e9aacdf6de30 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.571807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.590787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225a9ae-cf33-455c-b630-856f3f50ea24 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.634847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.635471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.635735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.636008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.637184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.638630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.640253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 179/358] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:04.644813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.664888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.669188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51af3-297c-4852-a6ae-e9aacdf6de30 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.712218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.712866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.713143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.713450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.714522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.715936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.717875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 180/359] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:04.718391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.764872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6969005-cbb9-4717-af63-6dd550636b47 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.805181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.805723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.806000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.806270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.806935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:04.807236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.808640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.818453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 180/360] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:04.818903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:04.841417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.845134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e4e4b-51d0-4967-966a-a83b7fa3d28a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.888305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.889331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.889612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.889881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.890550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:04.890856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.892250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8964d29-7936-4d7a-9581-4655b0fa01e7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:04.902342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 181/361] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:04.905967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.906260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.906678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.926552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:04.928149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:04.942021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a04b74-4c79-411e-9f35-dfd95180b389 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:04.957486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:04.988408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:04.989386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:04.989826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:04.990287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:04.997509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:04.998131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.000006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 181/362] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.000835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.007442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 182/363] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:05.008608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.093130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.137693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de71dc-1cfe-409a-8fd0-9bd060654093 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.155123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6b34e-4035-4e01-8967-aa155b075f40 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.173561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.174195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.174476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.174753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.175866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.177822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.184246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 182/364] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:04 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.184246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.197742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.198381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.198659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.198935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.200038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.201606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.208792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 183/365] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.208792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.235737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.292202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61102797-ee90-4dc0-9a73-9472db2fb1fa None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.311296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfde3c-c1dc-465d-ab44-60cbc99f8375 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.326575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.327197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.327478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.327753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] PATH_INFO: `/v3/users/4b5871589c96468182a85b46c20aa5cb` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.328902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] RBAC: Authorizing `identity:get_user(user_id=4b5871589c96468182a85b46c20aa5cb)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.330601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.332180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 183/366] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/users/4b5871589c96468182a85b46c20aa5cb => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.333315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.344969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.345525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.345802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.346067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.351314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.416740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.418198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.435522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ad8df-40d1-4697-b03c-08ea6fe6bd59 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.444859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.477005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.477599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.477879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.478154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.478877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=530e3f52da9144b3b7b0c97df02943f7) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:05.479178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.480609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.486094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 184/367] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.486620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.533202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 184/368] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:05.534418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.579059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.606879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3ba61-3da8-414c-9949-840cb863b4cd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.647100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.647712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.647994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.648264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.649173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=530e3f52da9144b3b7b0c97df02943f7) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:05.649478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.650953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.654987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1855e-d864-4f71-baa8-776e50b77321 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.657235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 185/369] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.658112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.688464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.699216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.699841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.700117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.700386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.701213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:05.701516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.702966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.716468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 185/370] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.717550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.788374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0985d-8590-443c-a961-1231ec58b91d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.872324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.872918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.873195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.873466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.899557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:05.903599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2ba2e-3773-493d-aa16-044fae163422 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:05.910677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:05.912592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 186/371] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:05.913419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:05.919334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:05.957027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:05.958176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:05.959894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:05.960400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:05.961407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] RBAC: Adding query filter params (role.id=1c84e2fcb51c4227a0edc690959c2fb1, user.id=4b5871589c96468182a85b46c20aa5cb, scope.system=all) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:05.962081 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:05.965004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:05.995284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 186/372] 10.222.0.16 () {56 vars in 1308 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/role_assignments?user.id=4b5871589c96468182a85b46c20aa5cb&scope.system=all&role.id=1c84e2fcb51c4227a0edc690959c2fb1 => generated 510 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:05.995284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.106992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6e7d1-a5c5-4ed8-810b-63b2f06481fb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.113143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.179317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.180058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.180342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.180613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.181851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.183342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.186376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 187/373] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:06.191932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.283608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833d8a2-363b-49dc-b38f-775a6a5fd99e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.294354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.341290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d4721-7147-405e-9b5c-2bce2c32d11c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.357016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.357638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.358100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.358378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.359457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:06.359761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.365531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.384788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 187/374] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:06.386521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.409337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.410030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.410312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.410585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.414936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.416482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.418544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 188/375] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:06 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:06.427146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.545666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f002f-f657-455c-8fa0-4974c8b2350c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.569026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.612355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.612635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.612995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.622951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce8c30-b42b-4d6c-85f4-192dc108a21b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:06.689270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:06.689881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:06.690155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:06.690489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:06.691223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:06.691583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.693339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.706271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 189/376] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:06 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:06.724642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:06.726331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:06.853323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 188/377] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:06.859481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:06.864700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:06.994599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0017beb-3941-4813-9ac2-01b016a2754a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:07.032073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:07.032706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.032989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.033257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] PATH_INFO: `/v3/projects/54beae32846044a3b8e04220f82543ad` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.034364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] RBAC: Authorizing `identity:get_project(project_id=54beae32846044a3b8e04220f82543ad)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:07.035820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:07.037854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:06 2022] GET /identity/v3/projects/54beae32846044a3b8e04220f82543ad => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:07.043133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:07.060839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:07.152172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ef650-1865-432d-ab69-b2585fe0e05a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:07.193114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dffb2d-f2ef-4f55-99e2-9f6cace16ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:07.193756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffb2d-f2ef-4f55-99e2-9f6cace16ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.194037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffb2d-f2ef-4f55-99e2-9f6cace16ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.194308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dffb2d-f2ef-4f55-99e2-9f6cace16ee6 None admin] PATH_INFO: `/v3/groups/b6bf2e383f6b42908e7341b5bc632790` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.205371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffb2d-f2ef-4f55-99e2-9f6cace16ee6 None admin] RBAC: Authorizing `identity:get_group(group_id=b6bf2e383f6b42908e7341b5bc632790)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:07.206828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dffb2d-f2ef-4f55-99e2-9f6cace16ee6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:07.209093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 189/379] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:07 2022] GET /identity/v3/groups/b6bf2e383f6b42908e7341b5bc632790 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:07.214784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:07.299471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:07.341224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5b41d-7ad6-4ee6-97a8-bebf751d56ea None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:07.384316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:07.384952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:07.385225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:07.385492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:07.386225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] RBAC: Adding query filter params (group.id=b6bf2e383f6b42908e7341b5bc632790, role.id=8a4cd71b629943809347939c4c950275, scope.project.id=54beae32846044a3b8e04220f82543ad) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:07.386525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:07.389640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2425b-e838-4590-ba75-7a3aae5b5545 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:07.397594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 191/380] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:07 2022] GET /identity/v3/role_assignments?group.id=b6bf2e383f6b42908e7341b5bc632790&scope.project.id=54beae32846044a3b8e04220f82543ad&role.id=8a4cd71b629943809347939c4c950275 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:08.517995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecea47-d593-4fad-a9f8-f99e3f39fcdf None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.518991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecea47-d593-4fad-a9f8-f99e3f39fcdf None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.519470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecea47-d593-4fad-a9f8-f99e3f39fcdf None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.521533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 190/381] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:08.526503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff623f6-ddd6-4709-8c9f-2bb55cca73e0 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.527345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff623f6-ddd6-4709-8c9f-2bb55cca73e0 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.527835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff623f6-ddd6-4709-8c9f-2bb55cca73e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.533234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c104d-6c33-45d3-98f7-f1ca2230c1e4 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.533739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c104d-6c33-45d3-98f7-f1ca2230c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.534229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c104d-6c33-45d3-98f7-f1ca2230c1e4 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.540772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 191/382] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:08.548583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6ff623f6-ddd6-4709-8c9f-2bb55cca73e0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.549587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1422ba-7dde-419a-92c8-989a65c66538 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.549976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1422ba-7dde-419a-92c8-989a65c66538 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.550335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6ff623f6-ddd6-4709-8c9f-2bb55cca73e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.550448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1422ba-7dde-419a-92c8-989a65c66538 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.552464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 192/383] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:08.553842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.554319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.554783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.573386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.574868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.643035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 192/384] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:08.644588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:08.644888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:08.645203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:08.648015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 193/385] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:08.672352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:08.673564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:08.733153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:08.742786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 193/386] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:09.002895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.012101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.061867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.133489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5242d4-b645-4cd9-a5cb-0e69559de380 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.167353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.168207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.168486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.168781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] PATH_INFO: `/v3/roles/member` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.180161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:09.180516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc405d3c-72e4-4dfb-a378-cccb1ac9ca8a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:09.185241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:09.186096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:09.187475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 194/387] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:09.188120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.230339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.231436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.232284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.232787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.246370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:09.252295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:09.253230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:09.254013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:09.256748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 194/388] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:09.257794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.277215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.308475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c74ae-df11-45c2-9ed6-742781dfbdde None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.333076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.349469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.350262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.350633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.350998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.362971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:09.369802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:09.370968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:09.371949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:09.376800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 195/389] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:09.377870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.385677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a177723-8e24-4aa9-9612-7001e67480c3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.433280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.435162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.435640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.436054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.436935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:09.437405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.438990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.451638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 195/390] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:09.453465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.525555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea21ad-1085-409a-ab98-c738df36b305 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.559966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.567748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bf847-3b76-49a1-89a5-d826391e4b04 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.569832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.570804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.570804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.570967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.571619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:09.571919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.573323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.586599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 196/391] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:09.586599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.590211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.611969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.612585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.612890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.613166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.613850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:09.614683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.616064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419770c1-aed9-420a-8770-b5fc17f7a9a7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.626046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 196/392] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:09.627131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7a2f-2623-425f-8c65-4de5a54fea14 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.627416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7a2f-2623-425f-8c65-4de5a54fea14 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.627687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc7a2f-2623-425f-8c65-4de5a54fea14 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.632800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 197/393] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:09.633818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7a2f-2623-425f-8c65-4de5a54fea14 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.696804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2120c2-00b5-4942-a46b-ee45ad0ca06e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.733551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.734305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.734590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.734870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.735972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.737434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.738450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc7a2f-2623-425f-8c65-4de5a54fea14 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.744142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 197/394] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:09.744142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.779933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a27b9-69b9-4cc2-961f-b7c2388e407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.780504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a27b9-69b9-4cc2-961f-b7c2388e407b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.780811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a27b9-69b9-4cc2-961f-b7c2388e407b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.781082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a27b9-69b9-4cc2-961f-b7c2388e407b None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.782191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a27b9-69b9-4cc2-961f-b7c2388e407b None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.783643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a27b9-69b9-4cc2-961f-b7c2388e407b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.790269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 198/395] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:09.790269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.790269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.790269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.806395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.807275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:09.808451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:09.855679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:09.860562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14980701-f2dc-434a-8260-056347e8a3b1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:09.884361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 199/396] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:09.885346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:09.912009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:09.913004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:09.913471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:09.913918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:09.915195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:09.916872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:09.923154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 198/397] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:09.924166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.017319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4628e2f-0816-4a58-a65e-65486003d248 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.031273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4f699-9f40-4969-b082-7dcfc974e663 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.056635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.059902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.060535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.060855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.061121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.062238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.063683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.070453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 200/398] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:10.070453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.074684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad2482-3d47-446d-b091-c7a9f52b6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.075324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2482-3d47-446d-b091-c7a9f52b6f0d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.075603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2482-3d47-446d-b091-c7a9f52b6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.075875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2482-3d47-446d-b091-c7a9f52b6f0d None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.077028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad2482-3d47-446d-b091-c7a9f52b6f0d None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.078484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad2482-3d47-446d-b091-c7a9f52b6f0d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.081295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 199/399] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:09 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:10.082332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c826b-a903-4566-bef0-2cd6175804c7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.082631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c826b-a903-4566-bef0-2cd6175804c7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.082956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c826b-a903-4566-bef0-2cd6175804c7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.084302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 200/400] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:10.088724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c826b-a903-4566-bef0-2cd6175804c7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.127031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.182677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a1472-38b8-418f-91fe-dcc60a060e92 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.197329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c826b-a903-4566-bef0-2cd6175804c7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.215978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.216739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.217016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.217284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.218401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.219811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.221439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 201/401] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:10.222162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.233643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cee40-8433-4e86-93ca-b0ec39097b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.234489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cee40-8433-4e86-93ca-b0ec39097b3e None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.234766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cee40-8433-4e86-93ca-b0ec39097b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.235037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cee40-8433-4e86-93ca-b0ec39097b3e None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.236940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cee40-8433-4e86-93ca-b0ec39097b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=530e3f52da9144b3b7b0c97df02943f7, group_id=8d4aeb8599e5471ca34a9e5c50162d90, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.238370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cee40-8433-4e86-93ca-b0ec39097b3e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.251100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 201/402] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 17 20:51:10 2022] PUT /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:10.252219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.252510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.252810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.272195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:10.273442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:10.328247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.355425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 202/403] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:10.356656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.376757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784f9f3-0443-4737-ac2b-08d4201c0b91 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.396741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.427554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.428320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.428658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.429026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.430929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc9c1d792224c5da0612bdf14e9d869, group_id=8d4aeb8599e5471ca34a9e5c50162d90, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.432416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.452081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 202/404] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 17 20:51:10 2022] PUT /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:10.453661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.495884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e5382-7127-4949-9908-13d9b81fbb23 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.539246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.540013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.540265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.540561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.553315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:10.558595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:10.559627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:10.560391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 203/405] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:10.569890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.611980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b1bd6-7fe0-4792-814c-4ccb33a522e5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.629620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.656771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.656771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.656771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.656771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.657180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=530e3f52da9144b3b7b0c97df02943f7, group_id=8d4aeb8599e5471ca34a9e5c50162d90, role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.657180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.675235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 203/406] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 17 20:51:10 2022] PUT /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/44957acdbea44978962b0cc8d2e0d069 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:10.696131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.749898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb10dd2-a85a-489d-b574-87110342b0dc None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.757117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:10.796693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.797370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.797651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.797915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.798590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:10.798891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:10.800325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:10.812119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 204/407] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:10.818161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.836756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d82126-1c77-4916-b693-d663fd6b49a4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.891035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:10.891604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:10.891878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:10.892158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:10.917773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:10.923109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:10.923952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:10.924728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:10.925308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 204/408] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:10.931056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:10.955800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948073b7-3435-4e0f-aed2-69045eee3e18 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:10.960365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:11.002826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.003464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.003740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.005110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.006423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.008296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.011193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 205/409] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:11.016585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.048951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:11.058045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b98fb4-308b-4687-8cb8-34f80067fc68 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.096309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.096946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.097328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.097651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.098464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:11.098761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.100319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.109289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 205/410] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:11.115077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.128402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3dd16-2f3c-4aa8-94a8-cf709b41d99a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.165577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.166483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.166954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.167399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.168866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.170474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.172455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 206/411] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:11 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:11.178376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.221099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6ed28-d1ce-4f8f-80cb-f09329a885c7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.224357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:11.270217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.271020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.271297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.271568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] PATH_INFO: `/v3/groups/b6bf2e383f6b42908e7341b5bc632790` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.273855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] RBAC: Authorizing `identity:get_group(group_id=b6bf2e383f6b42908e7341b5bc632790)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.275667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.278265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 206/412] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:11 2022] GET /identity/v3/groups/b6bf2e383f6b42908e7341b5bc632790 => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:11.287058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.297788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bf0f7-fc7c-4f88-83be-7eae505e736f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.340835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.341546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.341827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.342109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.345222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] RBAC: Authorizing `identity:create_grant(project_id=dbc9c1d792224c5da0612bdf14e9d869, group_id=8d4aeb8599e5471ca34a9e5c50162d90, role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.346669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.351165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:11.372380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 207/413] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 17 20:51:11 2022] PUT /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869/groups/8d4aeb8599e5471ca34a9e5c50162d90/roles/44957acdbea44978962b0cc8d2e0d069 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:11.473482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cc331-53c6-4306-a0ad-28819c8e37b5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.520103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bb256-0cc0-4416-bdc7-03a8e47945e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.521209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bb256-0cc0-4416-bdc7-03a8e47945e6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.521680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bb256-0cc0-4416-bdc7-03a8e47945e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.522121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bb256-0cc0-4416-bdc7-03a8e47945e6 None admin] PATH_INFO: `/v3/projects/54beae32846044a3b8e04220f82543ad` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.523400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bb256-0cc0-4416-bdc7-03a8e47945e6 None admin] RBAC: Authorizing `identity:get_project(project_id=54beae32846044a3b8e04220f82543ad)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.525001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bb256-0cc0-4416-bdc7-03a8e47945e6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.527147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 207/414] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:11 2022] GET /identity/v3/projects/54beae32846044a3b8e04220f82543ad => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:11.532470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:11.640605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:11.644868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8551aa-9dd8-49ff-92f5-88a79c3b845d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:11.682664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c76cb6-45fa-4ad6-831e-c36273d646ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:11.683595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c76cb6-45fa-4ad6-831e-c36273d646ae None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:11.684015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c76cb6-45fa-4ad6-831e-c36273d646ae None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:11.684427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c76cb6-45fa-4ad6-831e-c36273d646ae None admin] PATH_INFO: `/v3/projects/54beae32846044a3b8e04220f82543ad/groups/b6bf2e383f6b42908e7341b5bc632790/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:11.686524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c76cb6-45fa-4ad6-831e-c36273d646ae None admin] RBAC: Authorizing `identity:create_grant(project_id=54beae32846044a3b8e04220f82543ad, group_id=b6bf2e383f6b42908e7341b5bc632790, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:11.688107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c76cb6-45fa-4ad6-831e-c36273d646ae None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:11.704253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 208/415] 10.222.0.16 () {58 vars in 1442 bytes} [Sat Sep 17 20:51:11 2022] PUT /identity/v3/projects/54beae32846044a3b8e04220f82543ad/groups/b6bf2e383f6b42908e7341b5bc632790/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:12.774380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd053-e7b8-4e42-897a-d7e451f8e93a None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.775400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd053-e7b8-4e42-897a-d7e451f8e93a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.775883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd053-e7b8-4e42-897a-d7e451f8e93a None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.777724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 208/416] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:12.782886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb76c14-9d00-4c51-ae9e-cff122766f12 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:12.783828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb76c14-9d00-4c51-ae9e-cff122766f12 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:12.784298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb76c14-9d00-4c51-ae9e-cff122766f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:12.805249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2eb76c14-9d00-4c51-ae9e-cff122766f12 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:12.806746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2eb76c14-9d00-4c51-ae9e-cff122766f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:12.899069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 209/417] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:13.206152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce14348-3245-42c2-815e-5ac90316abae None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.206823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce14348-3245-42c2-815e-5ac90316abae None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.207328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce14348-3245-42c2-815e-5ac90316abae None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.209816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 209/418] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:13.214782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.215288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.215752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.234667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:13.236116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.277854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce14348-3245-42c2-815e-5ac90316abae None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.363614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 210/419] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:13.439484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:13.460814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce14348-3245-42c2-815e-5ac90316abae None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.507729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed374e-6ffd-42ec-8e47-f54bb808dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.508749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed374e-6ffd-42ec-8e47-f54bb808dde1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.509221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed374e-6ffd-42ec-8e47-f54bb808dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.509708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed374e-6ffd-42ec-8e47-f54bb808dde1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.523384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed374e-6ffd-42ec-8e47-f54bb808dde1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-d0ed374e-6ffd-42ec-8e47-f54bb808dde1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:13.528110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:13.529046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:13.530214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 210/420] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:13.535114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.573049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:13.593688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76e1a-596b-47e4-bb1d-632a4407cc35 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.593966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76e1a-596b-47e4-bb1d-632a4407cc35 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.594237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76e1a-596b-47e4-bb1d-632a4407cc35 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.595701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 211/421] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:13.611783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.612049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.612324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.642526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:13.643704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:13.727923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8d0e6-4850-4def-b126-9d74b1b57e0f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.736782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 212/422] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:13.755700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.808040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.808720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.809012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.809293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.810014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] RBAC: Adding query filter params (name=member) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:13.810319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:13.811687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:13.831424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 211/423] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:13.836968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:13.852108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:13.877897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7e771-1496-4669-bba5-084a99d391b5 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:13.919178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:13.919807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:13.920088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:13.920359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:13.932657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:13.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:13.938840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:13.939624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 213/424] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:13.945698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.026374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.062041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf0be3-1395-4f35-bb2b-85e5cf7d7d48 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.099857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cccd9-28c4-41f0-8a99-faad3c2ffd6a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.118239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.118871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.119246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.119525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.144263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.145709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.146241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.146883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.148190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] RBAC: Adding query filter params (name=member) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:14.148751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.150790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.164089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 214/425] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:14.165465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.169643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.204100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.205856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.303042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.307721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba7e8-8cb5-443f-a707-980f634c8060 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.330713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 212/426] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:51:14.337213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.347793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c57d62-98cc-4186-aeef-303dff470dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.348991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57d62-98cc-4186-aeef-303dff470dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.349265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57d62-98cc-4186-aeef-303dff470dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.349725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c57d62-98cc-4186-aeef-303dff470dc7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.361381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c57d62-98cc-4186-aeef-303dff470dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-88c57d62-98cc-4186-aeef-303dff470dc7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:14.366752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:14.367814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:14.368595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 215/427] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:14.369321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f53053-11ce-4349-a85c-2a7e5e532d44 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.369321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f53053-11ce-4349-a85c-2a7e5e532d44 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.369645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f53053-11ce-4349-a85c-2a7e5e532d44 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.377678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 216/428] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:14.377678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f53053-11ce-4349-a85c-2a7e5e532d44 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.439965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.453008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec14da-c50d-4a34-8714-e116c98f53a7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.481051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f53053-11ce-4349-a85c-2a7e5e532d44 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.488499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741f567-07ea-4474-88b3-1823eb48baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.489304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f567-07ea-4474-88b3-1823eb48baac None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.489938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f567-07ea-4474-88b3-1823eb48baac None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.490217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741f567-07ea-4474-88b3-1823eb48baac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.513326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.514084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.514366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.514634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.515729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.517200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.518799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 217/429] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:14.519548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.543252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.562422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f567-07ea-4474-88b3-1823eb48baac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.564134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741f567-07ea-4474-88b3-1823eb48baac None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.619027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fe9ea-0533-4fa1-8881-8753c1bf8060 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.647167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.653500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.654097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.654367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.654634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.655283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:14.655577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.656973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69752c4-d583-48ff-bbd3-eb180caa73cb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.660079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 213/430] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:14.664798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.664798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.664798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.668355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 218/431] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:14.669594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cef4e-e578-464a-a655-91243207de41 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.669876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cef4e-e578-464a-a655-91243207de41 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.670149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cef4e-e578-464a-a655-91243207de41 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.671455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 219/432] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:14.676062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cef4e-e578-464a-a655-91243207de41 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.681422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:14.682598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.746959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 214/433] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:14.747784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.761581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.788735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cef4e-e578-464a-a655-91243207de41 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.823195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.823792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.824120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.824401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.825672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.827154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.834120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 220/434] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:14.834120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:14.856396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb72714-45c5-458f-ac2a-812436fc210f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.892835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeacddf-c65d-4911-9754-aecdc8f9a3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.893454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeacddf-c65d-4911-9754-aecdc8f9a3cd None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.893832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeacddf-c65d-4911-9754-aecdc8f9a3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.894019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeacddf-c65d-4911-9754-aecdc8f9a3cd None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.895689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:14.896313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeacddf-c65d-4911-9754-aecdc8f9a3cd None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:14.897707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeacddf-c65d-4911-9754-aecdc8f9a3cd None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:14.899641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 215/435] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:14.900604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.900934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.901202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.919761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:14.920957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:14.942738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a43a1-2217-4a35-b293-eb24ecf0b87d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:14.977972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:14.978572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:14.978846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:14.979117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:14.990994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 216/436] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:14.991836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.015093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.075226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.076666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.111510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234bcb1-09ec-45eb-b320-c6e7e4baf04a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.135444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.147535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.148228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.148508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.148820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.149545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=530e3f52da9144b3b7b0c97df02943f7) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:15.149848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.151273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.168704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 217/437] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.168704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.200995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 221/438] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:15.202225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.245613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.273588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19257d4-c21b-44ac-8912-f9f37916d177 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.311918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.312607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.312861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.313135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.314279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.315802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.318281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 218/439] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.319091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.336505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7332ef2-c3aa-4204-9683-bb862a4e672d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.382908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.383826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.384104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.384500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.398058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:15.403158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:15.404066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:15.404892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 222/440] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:15.405541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.431654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.446367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1947e64-ecad-42ea-8a7d-e4c993dc9ee6 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.487368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.487990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.488267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.488537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] PATH_INFO: `/v3/projects/530e3f52da9144b3b7b0c97df02943f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.491245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] RBAC: Authorizing `identity:get_project(project_id=530e3f52da9144b3b7b0c97df02943f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.492721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.494413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 219/441] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/projects/530e3f52da9144b3b7b0c97df02943f7 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.495175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.507679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.526910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1359-7348-41cc-94d4-25f87566d620 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.567399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.568122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.568456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.568824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.580528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:15.585592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:15.586497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:15.587317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 223/442] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:15.587995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.611241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457a7b9-547d-4ac9-b329-2abedf79fe5e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.643875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.644470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.644771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.645079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.645915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=e5bac5bfcc784719ac8f886a28cb9df6, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:15.646217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.647646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.653115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 220/443] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=e5bac5bfcc784719ac8f886a28cb9df6 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.653880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.670794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.694556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff91df-0010-4b3c-9579-08941cfea5ec None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.734475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.735094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.735372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.735645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.736327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:15.736631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.738085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.752731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 224/444] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.752959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.753054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.782400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dd009-cbdb-42e4-8a18-5615c84559c3 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.819814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.820457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.820757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.821028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.822155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.823594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.825199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 221/445] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.825967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.864077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ac06e-af04-41ca-81ad-eb012b2d9d0c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.900645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.901349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.901636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.901887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.902551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:15.902848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:15.904234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:15.912741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 225/446] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:15.913534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:15.918622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:15.937825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c76e38-13c3-4f10-8654-dc671382bdf8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:15.970328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:15.970920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:15.971193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:15.971460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:15.984429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.013240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d35ea-a73e-4bde-845e-1e3969df3976 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.039389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.040900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.045793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.046378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.046651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.046926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.047639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=530e3f52da9144b3b7b0c97df02943f7) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:16.047939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.049399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.054955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 226/447] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=530e3f52da9144b3b7b0c97df02943f7&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:16.055713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.142613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.154855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 222/448] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:16.160629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.182827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ddbd-f558-4c3a-a79c-149636b5bc8b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.209122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.221747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.222359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.222679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.222964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.275969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18da9bb-5265-4178-bd2f-3c3740244f24 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.302266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.303718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.316480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.317671 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.317944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.318212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] PATH_INFO: `/v3/projects/dbc9c1d792224c5da0612bdf14e9d869` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.319427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] RBAC: Authorizing `identity:get_project(project_id=dbc9c1d792224c5da0612bdf14e9d869)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.321013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.323053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 223/449] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/projects/dbc9c1d792224c5da0612bdf14e9d869 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:16.328141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.388556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.435711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 227/450] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:16.441119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.458719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce01d4-6190-4569-a5e6-6f3a6f40f132 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.470315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.498994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.499616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.499888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.500155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] PATH_INFO: `/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.502705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] RBAC: Authorizing `identity:get_group(group_id=8d4aeb8599e5471ca34a9e5c50162d90)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.502910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.506431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 224/451] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/groups/8d4aeb8599e5471ca34a9e5c50162d90 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:16.512084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.557924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41d829-1959-4a2d-8cd2-a2976e64547a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.599566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.600190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.600463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.600813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] PATH_INFO: `/v3/projects/54beae32846044a3b8e04220f82543ad` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.602371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] RBAC: Authorizing `identity:get_project(project_id=54beae32846044a3b8e04220f82543ad)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.604488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.606363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 228/452] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/projects/54beae32846044a3b8e04220f82543ad => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:16.611522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.626580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155602a8-a3e3-4fc3-83d5-71ea2bf62e84 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.645417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.660958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.661673 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.661955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.662228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.662954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] RBAC: Adding query filter params (group.id=8d4aeb8599e5471ca34a9e5c50162d90, role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=dbc9c1d792224c5da0612bdf14e9d869) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:16.663288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.664736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.670608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 225/453] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/role_assignments?group.id=8d4aeb8599e5471ca34a9e5c50162d90&scope.project.id=dbc9c1d792224c5da0612bdf14e9d869&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:16.712434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a8b51-841e-4397-bd3d-b7bf24ed3a2c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.749051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85dddd-8d21-4e0d-90f9-2fe5bca7cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.749679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85dddd-8d21-4e0d-90f9-2fe5bca7cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.749954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85dddd-8d21-4e0d-90f9-2fe5bca7cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.750224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85dddd-8d21-4e0d-90f9-2fe5bca7cdd3 None admin] PATH_INFO: `/v3/groups/b6bf2e383f6b42908e7341b5bc632790` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.751375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85dddd-8d21-4e0d-90f9-2fe5bca7cdd3 None admin] RBAC: Authorizing `identity:get_group(group_id=b6bf2e383f6b42908e7341b5bc632790)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.752856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85dddd-8d21-4e0d-90f9-2fe5bca7cdd3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.755028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 229/454] 10.222.0.16 () {56 vars in 1180 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/groups/b6bf2e383f6b42908e7341b5bc632790 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:16.761107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:16.805493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:16.864904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2a1e2-c6a9-446e-b6d1-3c7d75c94c4e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:16.897884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:16.898493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:16.898758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:16.899025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:16.899744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] RBAC: Adding query filter params (group.id=b6bf2e383f6b42908e7341b5bc632790, role.id=8a4cd71b629943809347939c4c950275, scope.project.id=54beae32846044a3b8e04220f82543ad) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:16.900051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:16.901487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54297c-3c90-4018-b9c6-998537ec7d7e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:16.907235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 226/455] 10.222.0.16 () {56 vars in 1376 bytes} [Sat Sep 17 20:51:16 2022] GET /identity/v3/role_assignments?group.id=b6bf2e383f6b42908e7341b5bc632790&scope.project.id=54beae32846044a3b8e04220f82543ad&role.id=8a4cd71b629943809347939c4c950275 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:19.577455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a05cb-fe23-48ec-b4f0-36e82465d484 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.577958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a05cb-fe23-48ec-b4f0-36e82465d484 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.578030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a05cb-fe23-48ec-b4f0-36e82465d484 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.579837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 230/456] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:19.594628 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc45875-87ab-4236-b863-8884f4a1ffd4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.595140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc45875-87ab-4236-b863-8884f4a1ffd4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.595215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc45875-87ab-4236-b863-8884f4a1ffd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.619557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bcc45875-87ab-4236-b863-8884f4a1ffd4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:19.620790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bcc45875-87ab-4236-b863-8884f4a1ffd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:19.719169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 227/457] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:19.868749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9daad2-0eed-43ed-ae1d-4776d478f69e None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.869070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9daad2-0eed-43ed-ae1d-4776d478f69e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.869306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9daad2-0eed-43ed-ae1d-4776d478f69e None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.870648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 231/458] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:19.875873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.876164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.876441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.901687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:19.902884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:19.934517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:19.952391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f776e57-3259-4740-8930-b69ab1964136 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.952962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f776e57-3259-4740-8930-b69ab1964136 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.953462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f776e57-3259-4740-8930-b69ab1964136 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.955466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 232/459] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:19.961112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:19.961715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:19.962208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:19.994206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 228/460] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:20.012071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:20.013299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:20.093021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.114881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 233/461] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:20.235755 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03f507-7473-4bcc-b2a0-a43d587e3775 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.283700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.284649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.285166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.286368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.301657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.303355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-2862c071-eafa-4a5b-a395-19b6729aa330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:20.323913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:20.324890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:20.325723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:20.325723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:20.325723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:20.327523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 229/462] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:20.337835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.338421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab0c4-b969-42ee-8331-01fd73580336 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.338914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab0c4-b969-42ee-8331-01fd73580336 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.339422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fab0c4-b969-42ee-8331-01fd73580336 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.341483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 230/463] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:20.347005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.347534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.348551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.381017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:20.382222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:20.391313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:20.397816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:20.458531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 231/464] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:20.459689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.468698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202042d-3d13-438e-a9c3-d7f2e6ebcc1f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.509979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.510746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.511120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.511494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.512793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:20.513200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.515144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.545177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 234/465] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:20.546759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.587154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd90c80-3558-4d3f-b873-dddf5642b964 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.624319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.625765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.626234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.626727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.637741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.639526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.661092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:20.663417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de8fd9-c1b2-41f9-9359-cb5826805fad None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.666883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:20.670722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:20.671686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:20.672514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:20.672514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:20.672514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:20.674723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 232/466] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:20.675812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.716574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.717288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.717569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.717853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.732749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.734653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:20.751937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:20.752865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:20.753709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:20.753709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:20.753709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:20.753709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 235/467] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:20.753915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.800064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214852b-fc53-4031-9bb5-30fd348cf01f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.836447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.837371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.837650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.837930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.838571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.840001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.864568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 233/468] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:51:20 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:20.866273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.872140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3448c5-c654-4e99-9619-44c083b064ad None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:20.914979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:20.915790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:20.916163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:20.916535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:20.917570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:20.917977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:20.919522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:20.939604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 236/469] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:20.939604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:20.944321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:20.947660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.011558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66178271-7a85-45ad-98d1-0fd75f607d61 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.058839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.059816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.060417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.060792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.073716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.075201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4d62d-845d-4ca4-83a3-fd2c97c297cf None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.075662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:21.091370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:21.092709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:21.093539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:21.093539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:21.093539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:21.095059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 234/470] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:21.096223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.119505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.120354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.120804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.121244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.122041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:21.122485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.124006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.137639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 237/471] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:21.138570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.215157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80cdff-e4fb-42a0-9b86-2568a533cbec None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.227145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.236711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.256927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.257548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.257829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.258104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.258760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.260261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.279076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ab85f-d17f-4e3a-b7e8-08825ffe3b87 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.286514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 235/472] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:51:21 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:21.289345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.360578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.361139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.361415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.361690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.362355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:21.362654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.366293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc626df7-f27b-482e-9a5a-b0a5843eb884 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.382706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 238/473] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:21.384885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0c0d-6154-480c-aa0d-ea9e872862ef None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.385175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0c0d-6154-480c-aa0d-ea9e872862ef None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.385455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc0c0d-6154-480c-aa0d-ea9e872862ef None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.387209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 239/474] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:21.393538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc0c0d-6154-480c-aa0d-ea9e872862ef None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:21.468943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf71ac-7a17-4509-9b0b-998da440181f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.511718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799635e0-4425-441a-bab2-9eb2dead6243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.513021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799635e0-4425-441a-bab2-9eb2dead6243 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.513741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799635e0-4425-441a-bab2-9eb2dead6243 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.514216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799635e0-4425-441a-bab2-9eb2dead6243 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.515055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799635e0-4425-441a-bab2-9eb2dead6243 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.516730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799635e0-4425-441a-bab2-9eb2dead6243 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.537456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc0c0d-6154-480c-aa0d-ea9e872862ef None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:21.549074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 236/475] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:51:21 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:21.549074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.549074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.549074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.568617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:21.569997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:21.570691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.583041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:21.583927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:21.584425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:21.584953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:21.586362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:21.587862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:21.589592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:21.623551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 240/476] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:51:21 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:21.673051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 237/477] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:21.971357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.088701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc075732-cb13-4a31-8022-3a7358cbf5bd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.134435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.135084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.135362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.135634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.136346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.137761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.147373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.sql.core [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 20:51:22.147373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 20:51:22.147373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [parameters: {'id': 'ae36fedf91f744fe9576702d5549a2e8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 20:51:22.147373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81335) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result.read() Sep 17 20:51:22.167217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return ref.to_dict() Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:51:22.168146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application yield resource Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise self.value Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application session.commit() Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 20:51:22.169062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.session.flush() Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application compat.raise_( Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 20:51:22.169886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:51:22.170788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result.read() Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:51:22.173367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [parameters: {'id': 'ae36fedf91f744fe9576702d5549a2e8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:22.175361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 17 20:51:22.177431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:22.184059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 241/478] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 17 20:51:21 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:22.187351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.333398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec41f6-2267-43da-a104-be593d306de2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.382495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b95d1-122c-4eff-85a4-fe1cf688a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.383144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b95d1-122c-4eff-85a4-fe1cf688a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.383424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b95d1-122c-4eff-85a4-fe1cf688a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.383698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b95d1-122c-4eff-85a4-fe1cf688a0c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.409558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b95d1-122c-4eff-85a4-fe1cf688a0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9b0b95d1-122c-4eff-85a4-fe1cf688a0c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:22.414547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:22.415406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:22.416215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 238/479] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:51:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:22.423170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:22.503557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:22.581490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991ec0-475b-43dc-b767-c9ca6185e56d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:22.625554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:22.626077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:22.626355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:22.626630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:22.627292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:22.627592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:22.628985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5753de-4afd-4d25-a66c-d95bb1a0bf07 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:22.644127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 242/480] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:23.434438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c127-c194-4b19-a439-14fe67b4a7ba None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.435208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c127-c194-4b19-a439-14fe67b4a7ba None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.435208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031c127-c194-4b19-a439-14fe67b4a7ba None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.436585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 239/481] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:23.450433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279e86e-532f-44e4-9925-c548d3355468 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.450927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279e86e-532f-44e4-9925-c548d3355468 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.450999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279e86e-532f-44e4-9925-c548d3355468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.474471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5279e86e-532f-44e4-9925-c548d3355468 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:23.475660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5279e86e-532f-44e4-9925-c548d3355468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:23.562954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 243/482] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:23.939817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313089b5-480d-4dd0-82b3-ec11dc26b4a5 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.940204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313089b5-480d-4dd0-82b3-ec11dc26b4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.940398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313089b5-480d-4dd0-82b3-ec11dc26b4a5 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.944177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 240/483] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:23.949751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:23.950034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:23.950310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:23.951507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313089b5-480d-4dd0-82b3-ec11dc26b4a5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:23.965609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:23.977778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:23.979000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:24.038791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:24.110432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 244/484] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:24.154318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313089b5-480d-4dd0-82b3-ec11dc26b4a5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.226033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.226595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.226876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.227149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.241295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:24.246055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:24.246908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:24.247733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 241/485] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:24.253218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.432494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0e30a-1367-45b1-8154-ad418da04d71 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.456506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.480312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.480940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.481141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.481413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.482079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:24.482385 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.483755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.493552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 245/486] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:24.504858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.555467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:24.564208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:24.601568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e4f96-0fc8-4449-9c74-eefc8db96fb9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.630851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef99da96-cd33-4655-a585-7cd89eec81bd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.639298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97aa4f-9e5f-4271-a514-d5e3c5aaa520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.639921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97aa4f-9e5f-4271-a514-d5e3c5aaa520 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.640195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97aa4f-9e5f-4271-a514-d5e3c5aaa520 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.640469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97aa4f-9e5f-4271-a514-d5e3c5aaa520 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.650981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97aa4f-9e5f-4271-a514-d5e3c5aaa520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-6a97aa4f-9e5f-4271-a514-d5e3c5aaa520 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:24.655708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:24.656572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 242/487] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:24.663233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944606b-4581-4f76-b576-879a443eb016 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.663932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944606b-4581-4f76-b576-879a443eb016 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.663932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944606b-4581-4f76-b576-879a443eb016 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.663932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 243/488] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:24.663932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eac965-9882-4cca-b1d9-130e34374e64 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.663932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eac965-9882-4cca-b1d9-130e34374e64 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.663932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eac965-9882-4cca-b1d9-130e34374e64 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.664262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 244/489] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:24.665097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eac965-9882-4cca-b1d9-130e34374e64 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.676665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.677518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.677942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.678358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.689463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.690915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-2b361cf7-3025-43e2-b44e-b4b78324fb22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:24.706628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:24.707579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:24.708392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:24.708392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:24.708392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:24.712747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 246/490] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:24.713956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00098ad3-e407-4d2a-82d2-6be56d6cbee7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.714242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00098ad3-e407-4d2a-82d2-6be56d6cbee7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.714521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00098ad3-e407-4d2a-82d2-6be56d6cbee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.733208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-00098ad3-e407-4d2a-82d2-6be56d6cbee7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:24.734322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-00098ad3-e407-4d2a-82d2-6be56d6cbee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:24.774623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eac965-9882-4cca-b1d9-130e34374e64 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:24.816711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:24.816889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 247/491] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:24.816889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.816889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.816889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.823232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:24.823919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:24.824255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:24.824589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:24.825346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:24.825706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:24.827132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:24.833415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:24.834595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:24.837045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 245/492] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:24.837779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.842378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:24.921846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 248/493] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:24.923078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:24.987818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ae6bf-6cc2-4bd9-8edf-75a16df6c520 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.034320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede53d5c-9fb1-462e-abdd-5389f07cdba7 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.072844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.073955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.074467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.074986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.075403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.076081 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.076426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.076785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.077489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:25.077910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.079281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.087983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.090271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.092354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.106394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 246/494] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.107370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:25.111945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:25.112902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:25.113828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:25.113828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:25.113828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:25.115917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 249/495] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:25.117513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.168985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.234705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a975c-dc35-4916-a605-24daaf3e24e0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.236885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd772517-0b3d-4d5c-8620-4e560f7b116f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.275373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.276014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.276386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.276660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.276859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.277434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.277713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.277987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.284326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.285821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.288260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:25.293290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:25.294146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:25.294920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:25.298155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:25.298990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:25.301744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:25.302236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 247/496] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:25.302310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 250/497] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:25.302382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.302447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.359670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.405641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dced1a-2424-4927-8b41-ff94a99c8c10 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.420635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860ed40-916b-4623-af04-9b0e8129b065 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.435436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.441718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.442428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.442708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.443077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.453584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:25.457929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.458533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.458808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.459076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:25.459168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:25.459963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:25.461189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:25.462033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:25.462033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.462163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 251/498] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:25.462163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.467342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.486751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 248/499] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.486751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.575860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336fa41-10b4-483a-b2f3-fa742cbd77f3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.591442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.593608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88eb19a-c952-44eb-b552-e9888d5028b8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.613550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.614360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.614641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.614913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.615579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:25.615883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.617313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.625024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 252/500] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.625800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.627807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.628668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.629135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.629466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.630195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:25.630496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.631905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.668434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 249/501] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.669299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.699197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.725369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb7ce9-b0ef-4075-afaa-676c29b2c1f9 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.757497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.758095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.758369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.758640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.759294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:25.759593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.760985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846dc1c-f770-4702-8a99-3b45ea2531ff None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.767773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c3c90-ec5f-47fb-9d88-ba3c7a0cdeb8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.768508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 253/502] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:25.774134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7462d-0a55-46e5-aae8-852526007791 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.774134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7462d-0a55-46e5-aae8-852526007791 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.774134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7462d-0a55-46e5-aae8-852526007791 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.774134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 254/503] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:25.774134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7462d-0a55-46e5-aae8-852526007791 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.807676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.808304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.808584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.808889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.812714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.815923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.817792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:25.830757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:25.831683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:25.833122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 250/504] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:25.833689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.874622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7462d-0a55-46e5-aae8-852526007791 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.907111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.907727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.908065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.908273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:25.918588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:25.920026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:25.924076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:25.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:25.934663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:25.935486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:25.935486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:25.935486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:25.935643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91220bbc-4a01-4494-97e0-aae132081d6c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:25.935719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 255/505] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:25.935719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:25.974454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:25.975059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:25.975340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:25.975617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.027887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.038177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a181aae-9485-4e1e-b6eb-b2f872f0d959 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.043375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.044865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.070447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.071064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.071335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.071603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.082865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.084259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-33cdfeb7-007e-49a3-be80-d351deb23a5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:26.099106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:26.100192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:26.101240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:26.101240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:26.101240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:26.101240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 256/506] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:26.101499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.101715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.101983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.119250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:26.120427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:26.137365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.148874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 251/507] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:26.149989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.188274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 257/508] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:26.189421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.252585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff2fa6-aad5-435b-bd57-a932a389f4b6 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.268493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.291578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.292174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.292452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.292748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.293475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:26.293878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.295212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.298919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2f294-fad8-4800-b1d6-1a59360e6779 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.314521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 252/509] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:26.317135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.341809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.343284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.343803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.344511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.345533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:26.345841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.347346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.358281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 258/510] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:26.359063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.375984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.434258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872d613-fd49-4d7e-8751-58875915a30c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.466635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321343cc-3374-4d87-8563-2119bd551033 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.470007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.470606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.470884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.471157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.471794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:26.472092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.473777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.484742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 253/511] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:26.485533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.498878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.505378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.506001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.506274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.506546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.517251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.518715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:26.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:26.533621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:26.534719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:26.534719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:26.534719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:26.534719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 259/512] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:26.534719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.585709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906f23a-2510-4962-ab38-b1190fd7f20b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.594176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.619247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.619845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.620129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.620400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.632054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfc7ca-e716-4f55-9c76-abc8d63b6e37 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.664945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.665556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.665836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.666105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.673274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.674706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:26.686108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:26.687115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:51:26.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:26.688500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.688569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 260/513] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:26.688640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.688806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.735524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.784241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 254/514] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:26.785704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.792643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96404b42-0127-490a-8658-5aa23b63a69a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.797872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.826055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.826675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.826952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.827217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.837643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.839109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:26.853257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:26.854265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:26.855219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:26.855219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:26.855219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:26.856837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 261/515] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:26.857730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.886070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed99cb7-5839-4e1b-91f8-d607d5799c16 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.919271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.919881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.920159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.920423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.927706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.929163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:26.940246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:26.941148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 255/516] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:26.942533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:26.952408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:26.959453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6865342-02e1-4e15-8973-1720dd358386 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:26.994108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:26.994705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:26.994978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:26.995246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:26.995880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:26.996179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:26.997576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.008952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 262/517] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.009668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.009786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.046172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b38f3b2-1da5-48e0-a391-800fb7b40b4b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.078954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.079887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.080171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.080441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.081197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:27.081503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.083413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.100840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 256/518] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 17 20:51:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.101966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.116103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35624c42-ade6-48e9-9c6f-92c30ce0feb4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.149162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.149772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.150044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.150313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.160815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.162295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:27.177002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:27.177878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:27.178687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:27.178687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:27.178687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:27.178687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 263/519] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:27.178900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.193178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.198833 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a7146-49bb-4055-9782-1e2e9c65f4f0 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.225040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.234089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.234683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.234951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.235224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.235852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:27.236150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.237540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.248490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 257/520] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.249300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.283460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c7be2-55ed-4892-a816-c15693b37e2a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.315906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.316505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.316805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.317168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.317890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:27.318191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.319625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.344548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21caa3-9a71-40a1-975d-882f3d5423b1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.356080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 264/521] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.356921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.383865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.384419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.384715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.384987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.395709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.397197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:27.409409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:27.411235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 258/522] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:27.412094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.423810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.458130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7e0b0-ed1e-4fd4-9e7e-4af58855fb4f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.458888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.492286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.492906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.493181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.493452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.511275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fa993-aed1-44af-8a34-dfbc39053f45 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.543924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.544520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.544850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.545122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.545756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:27.546055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.547404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.558026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.559458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.559578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 259/523] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.560225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.622004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.656156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 265/524] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:27.656947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.657473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6892-86a9-4777-aaf8-f1b385f2e6cb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.666439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.694037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.694638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.694918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.695254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.695898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.697371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.717952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 260/525] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:51:27 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:27.718762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.760074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2ccb8-1085-441a-aad8-c46195f54091 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.799769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.800383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.800661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.800968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.841492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:27.847468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e909ad-3fea-4a89-9494-c8d5d11f0837 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:27.881974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.883602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.886437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:27.887048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:27.887326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:27.887601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:27.888705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:27.888705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:27.890064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:27.910963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 261/526] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:27.912475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:27.942980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.005267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 266/527] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:28.006428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.029100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39d0ed-76e2-48c4-94aa-af6222a99ef3 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.065493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.066499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.067251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.067935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.074020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.076012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.077504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:28.092047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:28.093649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:28.094510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:28.098353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 262/528] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:28.098906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.123413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463ddda-9aca-4054-955e-3c3066756e39 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.161742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.162330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.162611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.162878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.173373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.174939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.184899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:28.192140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:28.193083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:28.193924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:28.193924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:28.193924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:28.193924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 267/529] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:28.194150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.209828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f10bb3-12cc-4a56-899f-133ebb725227 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.247082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.248137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.248474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.248783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.249697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec86be5855254c8abc10fb636add2749) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:28.250022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.251449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.257659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 263/530] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/role_assignments?user.id=ec86be5855254c8abc10fb636add2749&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:28.258444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.314906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7926c09-dd92-4090-933e-cd46f1bd8f36 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.337688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.359289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.359923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.360199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.360577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.373800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.375284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:28.392644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:28.393530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:28.394382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:28.394382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:28.394382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:28.398845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12518e2f-c7bb-46a3-892a-b2241e6f4c48 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.399161 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 268/531] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:28.400113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.446091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.447127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.448270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.448822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.449740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:28.450760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.453220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.469818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 264/532] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:28.471589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.484041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.517245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a078933-e75c-404a-bd43-a0102d27b8bb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.561274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.562044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.562446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.562807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.563610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:28.564020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.566068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.584776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 269/533] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:28.585655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.602815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a04f4-72f0-4c87-bf3d-a915114bd000 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.618107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.650025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.651491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.652202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.652720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.653677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:28.654157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.655793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.675253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 265/534] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:28.677818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.697285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625b1be-f32c-4cf7-a740-094112c584ae None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.733238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.733945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.734288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.734632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.735522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=3f6f37bc49c64ed1a9a61accef657525) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:28.735934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.737643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.744495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 270/535] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/role_assignments?user.id=3f6f37bc49c64ed1a9a61accef657525&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:28.745551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.756827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.832431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20335e04-aa40-4663-a096-20d1c737c5da None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:28.894264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:28.895433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:28.896068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:28.897503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:28.915934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:28.917687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:28.946440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:28.947562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:28.948619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:28.952024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 266/536] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:28.960860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:28.963374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:28.968141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a1627-f923-42a9-9a48-2a1a6f5a5edd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.028941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.029579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.029857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.030127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.045825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.047298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:29.067914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:29.068905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:29.069715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:29.076699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 271/537] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:29.077704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:29.149257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b447f8-53dd-4631-9fcd-b95a493c8963 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.174278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:29.183095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.183697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.183977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.184246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.184965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:29.185264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.186694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.199882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 267/538] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:29.211678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:29.247923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237d690-8059-4fdb-846e-1ca347cbc4b7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.299807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.300417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.301285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.301735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.302481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:29.302782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.304237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.342651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 272/539] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:29.349696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:29.381031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:29.398714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75e71b-0aff-4bc7-9ceb-2e56c98180a0 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.459878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.460404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.460698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.460976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.461776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=82ecac0d9cc54fefbca5e3b6d81c31b9) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:29.462086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.463785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ef48d-d6aa-41be-862d-c6f0eba909a3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.481494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 268/540] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:29 2022] GET /identity/v3/role_assignments?user.id=82ecac0d9cc54fefbca5e3b6d81c31b9&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:29.538472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21345262-dc7c-47f3-905d-4b9c4c9adc29 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:29.583457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:29.584070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:29.584350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:29.584625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:29.585399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec498f2822ba4dc9b05a0866e885b5b8) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:29.585706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:29.587204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6abada-3d80-49e7-8d71-50907e992c0f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:29.593864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 273/541] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:29 2022] GET /identity/v3/role_assignments?user.id=ec498f2822ba4dc9b05a0866e885b5b8&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:30.134859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598c3e-e732-4612-b859-e7c3c3b49f9a None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.135366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598c3e-e732-4612-b859-e7c3c3b49f9a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.135441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3598c3e-e732-4612-b859-e7c3c3b49f9a None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.136951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 269/542] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:30.152812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.153330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.153330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.174488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:30.176206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:30.246192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 274/543] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:30.658848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598c3e-e732-4612-b859-e7c3c3b49f9a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.727749 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:30.794411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3598c3e-e732-4612-b859-e7c3c3b49f9a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:30.841851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:30.842500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:30.842775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:30.843044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:30.862140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:30.867507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:30.868377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:30.869194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 270/544] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:30.874710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:30.885020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.004245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377999e9-cc15-4a73-8ec4-a227061a2e92 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.040212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.040842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.041117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.041384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.042044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:31.042337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.043704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb1685-e9ba-4263-90b9-75727c35fe73 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.052951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 275/545] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:31.058630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.112805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe6b3b-911c-45fa-a006-410a0498b3a4 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.113196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe6b3b-911c-45fa-a006-410a0498b3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.114117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe6b3b-911c-45fa-a006-410a0498b3a4 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.115634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 276/546] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:31.123514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeb5f-363a-4433-9dd8-1cd9caeb89f2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.123793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeb5f-363a-4433-9dd8-1cd9caeb89f2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.124070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeb5f-363a-4433-9dd8-1cd9caeb89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.153853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-29afeb5f-363a-4433-9dd8-1cd9caeb89f2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:31.157089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-29afeb5f-363a-4433-9dd8-1cd9caeb89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.203756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580d454-671f-45ac-99c9-7d034749168c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.219289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.269841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.273219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.273857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.274131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.274396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.280641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 277/547] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:31.284829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c138242-da8c-4b57-8cda-b5386ca6b206 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.284829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c138242-da8c-4b57-8cda-b5386ca6b206 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.284829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c138242-da8c-4b57-8cda-b5386ca6b206 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.284829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 278/548] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:31.292242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.293998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.300458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.301471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.301967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:31.325119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:31.326249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:31.327328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:31.327328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:31.327328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:31.329311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 271/549] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:31.335177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:31.335309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.336387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:31.439539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a487e-7cfd-479e-a6bf-ea4e40beeac7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.475481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.476088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.476366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.476634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.477295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:31.477596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.479403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.493657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 272/550] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:31.499145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.506171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 279/551] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:31.525078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.605764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da8d8b-2449-47de-985e-4ec2b7384677 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.640600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.641253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.641527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.641797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.649651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.651098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:31.663569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:31.664664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:31.666959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 273/552] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:31.670222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.670716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.756468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.777084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295dae1-320a-40fe-a44c-d13c70fd30b3 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.813827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.814688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.815116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.815538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.816794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:31.817241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:31.818814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:31.845829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 274/553] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:31.852142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.852280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124a0b7-3ea3-4ff4-a974-e0197e76b6f9 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.892378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.899208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:31.900065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:31.900636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:31.901222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:31.912593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:31.917930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:31.919075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:31.920024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:31.922236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 280/554] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:31.923136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:31.962649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018f367-3ff5-47bc-ae1b-fede4ee8e8e9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:31.999608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.000518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.000991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.001653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.014156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.037381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf379f-4979-4002-bc14-ea8bd2b3b30f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.075749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.076344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.076623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.076970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.083988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.085897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.088642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:32.093972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:32.095071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 281/555] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:32.095930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.149778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.202737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 275/556] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 17 20:51:32.202737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.209121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab892898-a5a5-48c9-9976-7d04df370a96 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.247459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.249802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.250975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.251256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.251534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.252404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:32.252736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.255673 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7e939-91e0-4846-a9e5-d99343284328 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.264707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 282/557] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:32.270552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b494b-e991-4c26-8f34-ced0a3fdaa56 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.271480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b494b-e991-4c26-8f34-ced0a3fdaa56 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.272171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b494b-e991-4c26-8f34-ced0a3fdaa56 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.273903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 283/558] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:32.275090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b494b-e991-4c26-8f34-ced0a3fdaa56 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.310988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af8e5-005f-4aa4-b0b0-24920cbe88f9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.348650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.349280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.349553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.350267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.350468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:32.350765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.352123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.360296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 276/559] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:32.361116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.393533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b494b-e991-4c26-8f34-ced0a3fdaa56 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.406620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.428438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.429090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.429369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.429641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.439922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.442151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-0fec30be-f633-4a53-9db3-9545e5e6ddc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:32.456103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:32.457146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:32.458031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:32.458031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:32.458031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:32.458031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 284/560] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:32.458848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.459122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.459391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.464958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d42147-7a05-456e-9d4e-f7b8f8240bcd None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.466481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.476726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:32.477904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.500521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.501192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.501475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.501749 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.513075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.514518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9bb442e0-e0d2-4402-bf83-c2dae7aa513e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:32.528573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:32.529656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:32.530496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:32.530496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:32.530496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:32.530496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 277/561] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:32.531057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8e3e7-ec5f-468f-b11e-0ca145dfa3e3 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.531149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8e3e7-ec5f-468f-b11e-0ca145dfa3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.531348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8e3e7-ec5f-468f-b11e-0ca145dfa3e3 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.532579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 278/562] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:32.533291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8e3e7-ec5f-468f-b11e-0ca145dfa3e3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.548723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 285/563] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:32.549568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.645934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.648060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8e3e7-ec5f-468f-b11e-0ca145dfa3e3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.654410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9730ad-281a-4e88-af64-c2b413b88fda None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.683715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.684344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.684623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.684927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.694061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.694668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.694948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.695226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.695866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:32.696174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.696904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.697621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.698695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.710264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 286/564] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:32.711066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-3f09a4e1-4e7a-42a6-b8db-004c5b9046dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:32.714276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:32.715378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:32.716596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:32.716596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:32.716596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:32.716596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 279/565] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:32.716596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.716932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.717103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.730267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.735838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:32.737061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:32.816700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 280/566] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:32.816700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.832967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488cc2-3d71-4c8c-b188-ce4bcbc14971 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.868431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.869068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.869343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.869615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.870246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:32.870543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.872326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.883951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 287/567] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:32.885051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.900661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.922950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d5be-3251-4829-ad0c-d8729069a2bd None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:32.958699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:32.959324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:32.959599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:32.959871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:32.962874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:32.967801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:32.969916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:32.982365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:32.983218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:32.984072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 281/568] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:32.984536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:32.996906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbd364-23d7-48d1-8533-b7db15543e2b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.030495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.031094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.031366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.031635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.032267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.032563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.033954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.046331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 288/569] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.051379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.094995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a70e7c-02ec-4cee-949f-693811e8eded None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.130525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.131151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.131431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.131703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.132549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.142248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:33.147679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:33.148642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:33.149666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 282/570] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:33.150390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.160849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0684f-a3a4-4468-92b1-5a1605db9004 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.193925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.194538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.194810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.195083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.202585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.204140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.206195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:33.216249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:33.217748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 289/571] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:33.218682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.253615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353339-0c8f-4a32-b212-f1317cc884b7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.287893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.288490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.288795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.289069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.289767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.290062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.291470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.302822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 283/572] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.303480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.328459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fc6f5-8a86-41b0-9243-c21899639e47 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.363907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.364543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.364854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.365124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.366992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.373584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.375039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:33.387715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:33.388767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:33.389703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 290/573] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:33.390272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.412319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344eb72-6910-4e70-b11b-ace3d64d7185 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.436646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.458578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.459218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.459499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.459784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.470040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:33.475723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:33.476750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 284/574] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:33.477822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.503343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90f2c9-abac-4621-ab20-3aff4a661925 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.548494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.549179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.549456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.549733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.550403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.550847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.552518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.562321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 291/575] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.563032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.585355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c94d8-30e7-4f03-a496-2ff81a1ad494 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.622261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.623787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.624070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.624345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.625111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.625416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.626895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.638316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.657962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 285/576] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.657962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.686682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.702932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39089008-d5c0-484b-9b43-366673237f2f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.754541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.755248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.755522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.755789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.758757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=67e76f823b3a48368f5476c36acb9341) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.759060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.760505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.766464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 292/577] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/role_assignments?user.id=67e76f823b3a48368f5476c36acb9341&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.767361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.785115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929dde8-f1f0-4497-87c5-5da9b39cdc04 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.832430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.833105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.833383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.833656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.834410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.834731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.836184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.857075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 286/578] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.858280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.905565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321cbd8-3423-495e-b451-e3340f18b7de None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:33.933762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.951950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:33.952553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:33.952862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:33.953891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:33.955522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:33.955522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:33.957599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:33.971963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 293/579] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:33.971963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:33.985894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:33.997993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07229f-7015-4dbb-86a8-9c5f7ef0d423 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.049577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.050272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.051257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.051722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.065018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.067320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:34.088546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:34.089935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:34.090896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:34.090896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.090896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:34.090896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 287/580] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:34.090896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.098679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79617-e3c7-4144-8d06-fabd0aff4ac8 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.135964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.136597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.136933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.137239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.149101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.151077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:34.166790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:34.167860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:34.168765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:34.168765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.168765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:34.172751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 294/581] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:34.173656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.212584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.234152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780efcf-1a7c-4cee-aabe-cb476f207f62 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.271290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.271941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.272174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.272445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.283238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.284717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.288156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.302642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3ee34-4940-4db9-8a63-2f9e92fc7a4e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:34.303387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:34.304294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:34.305234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:34.305234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.305234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:34.305234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 288/582] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:34.305458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.344086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.344741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.345021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.345292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.356947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.358665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:34.374615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:34.375542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:34.376399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:34.376399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:34.376399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:34.377951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 295/583] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:34.378750 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.427724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305dd8e-efe3-48cb-9b88-c70071e7bd91 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.462912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.463434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.463712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.463984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.464624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:34.464944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.466329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.478835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.484201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 289/584] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:34.484201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.497936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f919b04-c119-475c-a9dc-3924f398ad93 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.537058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.537654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.537928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.538195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.538835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:34.539130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.540638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.547793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.557519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 296/585] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:34.557519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.601504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e0e3-89d9-41d4-a481-0492a2e30851 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.636528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.637208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.637482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.637807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.638449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:34.638745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.640124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.652506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 290/586] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:34.653396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.665465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f0f5e-b814-4b2b-947a-c8bf9e8f8cb4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.702209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.702809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.703084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.703455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.703981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:34.704276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.705924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.723653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 297/587] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:34.723653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.731155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.773042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff94a2-7a2c-4dd3-a215-858b37794b73 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.792436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.810992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.811619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.811899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.812175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.819590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.821032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.826299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f074fa-8088-4abf-a9f6-eb5f8799b2b2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:34.833251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:34.834219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 291/588] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:34.835147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.865107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.865724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.866002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.866272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.872991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.874434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:34.886985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:34.887962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 298/589] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:34.889120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:34.948913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc5796-b942-4e30-a3ac-f523930eaa03 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.960932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:34.986135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:34.986688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:34.987082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:34.987363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:34.995385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:34.996377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c578ca6-8642-4018-92b6-188704136aa6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:34.996853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:35.009559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:35.010627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 292/590] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:35.011501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.037093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.037709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.037986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.038258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.045931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.047378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.049087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:35.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:35.062024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:35.062956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 299/591] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:35.063556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.123154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98449a31-0555-4343-8c23-f5e802e0106f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.156633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.157259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.157537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.157812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.158539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:35.158838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.161149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.183410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 293/592] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:35.183410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.194043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a611b-61aa-4afc-b444-78c305289329 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.219149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.229182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.229868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.230148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.230448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.231109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:35.231414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.232902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.247270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 300/593] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:35.252643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.281291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.292741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f615519-5b46-4bd3-a378-6702d4dd0e7f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.329788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.330394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.330675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.330943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.331633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:35.332613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.334275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.357488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 294/594] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:35.357488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.367576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2cf01-e720-406f-8f5d-ccce925728e3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.404517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.405152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.405427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.405698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.406423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:35.406785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.408332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.426403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 301/595] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:35.427218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.473077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.490850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7516a-8ba2-42b3-a137-abf586490caa None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.527225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.527414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea77ef-9666-4cea-bbbe-5a04f95d653e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.527768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.528042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.528311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.539929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.541517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:35.556370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:35.557421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:35.558279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:35.558279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:35.558279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:35.558279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 295/596] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:35.558279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.566243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.567153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.567430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.567703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec86be5855254c8abc10fb636add2749/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.569246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.585372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=ec86be5855254c8abc10fb636add2749, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.586835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.600346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 302/597] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:35 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec86be5855254c8abc10fb636add2749/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:35.601218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.687073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a179c2-20f6-4e9e-92f9-1234cb792c6e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.731198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.731870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.732144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.732410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/3f6f37bc49c64ed1a9a61accef657525/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.741140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.750729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b79ce-13aa-4376-ad3c-5fa6f7705135 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.754765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=3f6f37bc49c64ed1a9a61accef657525, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.759636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.774740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 296/598] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:35 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/3f6f37bc49c64ed1a9a61accef657525/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:35.775663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.809305 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.810338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.810839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.811416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.826192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.828283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:35.849888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:35.850929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:35.851818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:35.851818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:35.851818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:35.853249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:35.860059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 303/599] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:35.861908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.911795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3981a-ad6d-4738-9cfa-64c6cd54fce0 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:35.958683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:35.959314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:35.959594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:35.959862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:35.960530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:35.960885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:35.963072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:35.978527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 297/600] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:35.986389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:35.994499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7c76-4cd5-4f84-8ea6-b7b589ce9932 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.041505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.042918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.043477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.043922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.044919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:36.045781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.047628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21736167-2219-4f4e-8a1e-2a64c9509122 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.065961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 304/601] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:36.068303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de9c11-056b-44e3-8870-e5e5178be0eb None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.068782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de9c11-056b-44e3-8870-e5e5178be0eb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.069756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de9c11-056b-44e3-8870-e5e5178be0eb None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.071468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 305/602] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:36.072331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de9c11-056b-44e3-8870-e5e5178be0eb None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.083164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.122143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecfaa5-02ff-4f56-9517-c40d42061dae None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.149510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.181171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.181819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.182092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.182369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.189851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.192273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.202286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de9c11-056b-44e3-8870-e5e5178be0eb None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-15a880ad-abfb-4053-9b06-4fc11511e120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:36.204710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:36.205558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:36.206424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 298/603] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:36.207472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.207635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.207937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.228757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:36.230029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:36.245993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.246968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.247460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.247948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.256080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.257950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:36.271846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:36.272791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:36.273678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:36.276033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 306/604] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:36.277147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.303335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 299/605] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:36.304202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.402238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.453343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a9e2-9bc1-41ad-ab93-65e4e58dfd2d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.458681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ce57c-a07b-43f1-8eaf-8c0d4b4920c3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.473122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.511849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.512463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.512773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.513050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.513750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:36.514051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.515486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.526263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.526871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.527147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.527418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.528113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:36.528414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.530024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.542235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 300/606] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:36.548011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.552641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 307/607] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:36.556129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.683785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14014543-0a3c-428f-ac24-2d2384422967 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.699360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6db23b-96da-45da-b266-bcccb24202df None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.721707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.722989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.723337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.723689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec498f2822ba4dc9b05a0866e885b5b8/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.734707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.740557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=ec498f2822ba4dc9b05a0866e885b5b8, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.742677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.748368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.749099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.749503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.750625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/82ecac0d9cc54fefbca5e3b6d81c31b9/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.759697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 301/608] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:36 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec498f2822ba4dc9b05a0866e885b5b8/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:51:36.760922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:36.770333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=82ecac0d9cc54fefbca5e3b6d81c31b9, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:36.772591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:36.798067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:36.801000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 308/609] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:36 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/82ecac0d9cc54fefbca5e3b6d81c31b9/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:36.913361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30464a9d-77d0-4d01-93fa-3f4124f0dfe0 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:36.958322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:36.959008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:36.959289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:36.959562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:36.973145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:36.978524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:36.979502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:36.980344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 302/610] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:36.985631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.113163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064787c-9c1d-4d69-ab14-41a18bd67546 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.153756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.154365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.154640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.154912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.155581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:37.155880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.157283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.169079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 309/611] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:37.174718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.293676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:37.342888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a059275-9f2a-4674-ac79-786e1acb269b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.398629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.399573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.400051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.400582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.412111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.413894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:37.438840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:37.439816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:37.440708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:37.440708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:37.440708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:37.443099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 303/612] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:37.453771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.523031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:37.612375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08679e-3b51-464b-a8fb-c773032df345 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.662352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.662961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.663240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.663509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.664152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:37.664453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.665848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.679068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 310/613] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:37.684609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:37.872742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f380715-6b3e-4ddb-a3ad-21d26262099b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:37.922126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:37.937860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:37.938734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:37.939017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:37.939289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:37.953733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:37.955448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:37.970604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:37.971547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:37.973779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 304/614] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:37.977946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.142043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.145716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a5aa4-a0f5-4acc-9d5c-e11c490f4313 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.205410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.206022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.206295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.206597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.207637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:38.207933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.209529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b3d24-dbff-473d-a51f-53c6d17ca223 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.232503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 311/615] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 17 20:51:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:38.238721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.385253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6300a2b-33eb-4da8-8b6a-c4740567c575 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.398428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02638d-d6a3-4deb-a6f3-b95ba61c5160 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.398709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02638d-d6a3-4deb-a6f3-b95ba61c5160 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.398899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02638d-d6a3-4deb-a6f3-b95ba61c5160 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.400543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 312/616] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:38.408174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f44fa-de71-460b-a35c-a98f32650399 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.408465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f44fa-de71-460b-a35c-a98f32650399 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.408942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f44fa-de71-460b-a35c-a98f32650399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.432844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9c0f44fa-de71-460b-a35c-a98f32650399 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:38.434342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9c0f44fa-de71-460b-a35c-a98f32650399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.457792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.458444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.458725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.458996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.485731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.487824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.513029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 313/617] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:38.513531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:38.514403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:38.515258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:38.515258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:38.515258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:38.517395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6444-85c7-43af-98cc-14b77c48b46e None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.517936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6444-85c7-43af-98cc-14b77c48b46e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.518409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f6444-85c7-43af-98cc-14b77c48b46e None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.520827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 305/618] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:38.525694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.526440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 314/619] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:38.530020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877987-0e0c-464f-9887-a64e37030746 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.530532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877987-0e0c-464f-9887-a64e37030746 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.531264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df877987-0e0c-464f-9887-a64e37030746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.561352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-df877987-0e0c-464f-9887-a64e37030746 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:38.565807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-df877987-0e0c-464f-9887-a64e37030746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:38.613116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.683083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a52d7-cc1d-4a67-85a7-839ccf3af37b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.733905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.734982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.735620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.736093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.737148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:38.737652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.744698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 315/620] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:38.746009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:38.763103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 306/621] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:38.768340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877987-0e0c-464f-9887-a64e37030746 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.801301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.833705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:38.884897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df877987-0e0c-464f-9887-a64e37030746 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.962694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:38.963712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:38.964199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:38.964662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:38.965816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:38.969344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb14b0-2f9b-43c3-b7e4-e79ee409fb86 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:38.981730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:38.983311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:39.007733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:39.008774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:39.009643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:39.014188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 316/622] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:39.014996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.015844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.016369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.016846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.020292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.042839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:39.047814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:39.048727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:39.049578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:39.057303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 307/623] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:39.058175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.115748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.145021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b7804-b62a-47e0-a63b-a1960c1c3593 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.194267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.195411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.195867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.196524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.197795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:39.198301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.199921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.213564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93656e29-5ddc-4d95-bec6-8fdb23ae3f8b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.217743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 317/624] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:39.218769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.256621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.257376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.257789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.258058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.258851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:39.259307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.260759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.274422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 308/625] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:39.275744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.301850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.352562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c14fdf-d7ef-4a02-b072-770436d49f77 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.400665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.401496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.401775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.402050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.406594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.417222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:39.417764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927372-e757-4d82-9d14-6d4dafbbb285 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:39.423943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:39.425178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:39.426558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:39.430574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 318/626] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:39.431440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.464037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.464797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.464958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.465203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/67e76f823b3a48368f5476c36acb9341/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.481237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=67e76f823b3a48368f5476c36acb9341, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.482672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.501609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 309/627] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:39 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/67e76f823b3a48368f5476c36acb9341/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:39.502991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.545247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eee9c-f4f4-41fc-a51b-0f0088442ea9 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.588285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.588997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.589275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.589599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.603492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.605061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.620699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-63b76b57-f58a-48ca-aa4b-e1c04431dad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:39.621287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:39.622284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:39.623154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:39.623154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:39.623154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:39.623154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 319/628] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:39.624700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d136-fd8c-4d73-bf8d-af9f2b3a1b62 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.624700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d136-fd8c-4d73-bf8d-af9f2b3a1b62 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.624996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57d136-fd8c-4d73-bf8d-af9f2b3a1b62 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.630727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 320/629] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:39.631663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d136-fd8c-4d73-bf8d-af9f2b3a1b62 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.664490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6049277-dc1e-4bde-9c9e-6c98c4e8501a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.701088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.708116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.708755 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.709714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.709714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.709974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:39.710265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.711639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7294f-b3c3-4674-b850-a1196cc01f0b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.719950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 310/630] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:39.721145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b17b-0141-4fc2-a528-3180baef8acc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.721434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b17b-0141-4fc2-a528-3180baef8acc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.721713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b17b-0141-4fc2-a528-3180baef8acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.741676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1362b17b-0141-4fc2-a528-3180baef8acc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:39.742867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1362b17b-0141-4fc2-a528-3180baef8acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:39.778848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57d136-fd8c-4d73-bf8d-af9f2b3a1b62 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.820706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 311/631] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:39.820706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0469f4-bcd4-43e6-93d3-38249ced58c7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.820706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0469f4-bcd4-43e6-93d3-38249ced58c7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.820706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0469f4-bcd4-43e6-93d3-38249ced58c7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.820706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 312/632] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:39.820706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0469f4-bcd4-43e6-93d3-38249ced58c7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.823637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.824254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.824534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.824852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.825566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:39.825868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:39.827252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75400b0b-276c-4aba-8af5-0875922c841a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:39.846914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 321/633] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:39.849135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.849489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.849836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:39.869635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:39.871007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:39.893637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:39.935935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0469f4-bcd4-43e6-93d3-38249ced58c7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:39.966151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 322/634] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:39.967382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:39.997513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:39.998085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:39.998359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:39.998625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.033581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.035834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.048638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:40.069466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:40.070461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:40.071388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:40.071388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:40.071388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:40.073418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 313/635] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:40.078829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.206137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348be4a-7530-45b2-bf18-375c44f2fe64 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.211388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f221a92-f18c-4ed0-9da1-f9eff6579a99 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.235156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.249093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.250268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.250734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.251190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.252017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:40.252498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.254471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.267543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 314/636] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:40.268710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.309147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.309776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.310055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.310319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.318985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.320479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:40.333593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:40.334590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:40.335537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:40.339489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 323/637] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:51:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:40.340380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.386759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e667c8e-b9b3-4419-9d62-7027d2ff3ec8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.431071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.431729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.431955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.432246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.445994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:40.452529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:40.453579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 315/638] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:40.460489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.482113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.484989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad6bec-5271-4d9e-a045-9d123ef3ccf4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.511556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.528233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.528890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.529175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.529431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.537858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.539543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:40.551137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:40.552489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 324/639] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:40.553867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.589218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a271a-1f60-40b7-9db9-e5a3d8109c23 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.625104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.625779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.626065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.626341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.637213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:40.642387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:40.643509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 316/640] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:40.645223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.662228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6e1cb-d52c-4208-8f2c-e677a2c4a29f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.696093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.696693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.696988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.697272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.697965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:40.698260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.699720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.722930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 325/641] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:40.723798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.742930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.751452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.758405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de059b8-38d0-4ab2-b53e-aca7119d0e46 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.798769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.799287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.799568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.799842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.800553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:40.801134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.802980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.812046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 317/642] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:40.812718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.829177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06d91-9bb1-4602-8f32-8a42f3212974 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.865646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.866244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.866519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.866783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.867497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:40.867796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.869263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.887240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 326/643] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:40.887866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.924144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a8c52-6bd2-468f-8e25-c7b9c9178bb8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.959448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:40.960151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:40.960423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:40.960715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:40.961376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:40.961668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:40.963069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:40.965662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:40.977270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 318/644] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:40.978128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:40.990454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4775c0-594a-4154-b424-89a6014640fb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:40.999947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.026362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.027108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.027388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.027669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.084521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1abc9-1259-4ab6-9f2d-4bd76b8f9c3b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.099459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.100958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.121210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.121836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.122116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.122389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.134139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.135593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:41.150797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:41.151763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:41.152845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:41.152845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.152845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:41.152845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 319/645] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:41.152845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.185392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.220171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 327/646] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:41.220171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.236231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.270586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883a8a8-8d53-42e6-9ef9-7b153e791803 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.306926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.307915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.308431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.308906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.323202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260be520-ab6b-4faf-b9de-3ff24da54fc5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.357827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.358371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.358644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.358924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.373375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.373375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.383087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.384202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:41.390461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:41.391390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:41.392256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:41.392256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.392256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:41.392256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 328/647] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:41.392256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.422361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.462517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.499078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 320/648] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:41.499078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.507590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0a6b-0fb1-4e17-9b4f-68b8660188b3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.542108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.542753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.542963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.543230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.543861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:41.544159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.545562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.563867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 329/649] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:41.564872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.601912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ccbd4-24cb-4149-a663-e476fff5fa27 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.640136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.640727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.640997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.641293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.654448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.655923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.661489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:41.672418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:41.673390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:41.674282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:41.674282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.674282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:41.674282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 321/650] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:41.674513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.683599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b08276-2ada-4c60-ae33-d1a80bf0e146 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.702862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.718677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.719285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.719553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.719822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.720457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:41.720780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.722717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.739621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 330/651] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:41.739621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.786159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a710d45-6af7-4799-8359-566fe005d7df None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.819745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.820371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.820651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.820958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.831473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.832883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae15bd4-064b-4cde-8740-5085ea578616 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.833001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:41.846620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:41.848149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:41.849055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:41.849055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:41.849055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:41.849055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 322/652] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:41.849271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.871333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.871954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.872230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.872503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.880249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.881708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:41.892657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:41.893578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 331/653] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:41.894876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:41.903372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.928031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:41.950098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd662c-6613-460f-b578-7929a439da78 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.982990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:41.983599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.983881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:41.984156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:41.984835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:41.985137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:41.986496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6682a0-22da-4acb-b9b7-d88b3172a96b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:41.995926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e5838-d731-4f99-924f-358560d44f2e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:41.998445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 323/654] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:41.999600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534129c-8f3f-4916-8ab0-71cc4d9f1b7d None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:41.999884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534129c-8f3f-4916-8ab0-71cc4d9f1b7d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.000161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534129c-8f3f-4916-8ab0-71cc4d9f1b7d None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.001685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 324/655] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:42.002389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534129c-8f3f-4916-8ab0-71cc4d9f1b7d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.032081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.032732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.033009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.033283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.041620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.043057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:42.055291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:42.056286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 332/656] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:42.057229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.107646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534129c-8f3f-4916-8ab0-71cc4d9f1b7d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.123987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.141585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.142183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.142456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.142770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.143346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:42.143638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.145185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.147344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.156887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 325/657] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:42.157680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.164446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2bd9a-8cf2-43d1-bfc3-56438174921f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.198068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.198663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.198938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.199429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.199925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:42.200226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.201804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed73e89-8279-4bff-b2e1-17d5e010bace None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.219202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 333/658] 10.222.0.16 () {56 vars in 1139 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:42.220373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.220651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.220965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.240803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:42.242029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:42.258304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d99e1-a8dc-4219-9a2d-e453d62aeff2 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.290148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.290760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.291034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.291304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.297919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.299347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.307343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 334/659] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:42.308168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:42.310856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:42.311720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:42.312568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 326/660] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:42.314242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.367228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.386396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.420200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68d33-3d13-467a-9b61-aa6d0e77ffc4 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.433652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120549e-2050-456d-90b4-81c2e84fe4ce None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.457420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.458027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.458310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.458580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.459312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:42.459614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.461069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.469519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.470627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.471048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.471457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.478992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 335/661] 10.222.0.16 () {56 vars in 1137 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:42.479346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.479812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.481222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:42.495601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:42.496500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 327/662] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:42.498013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.590891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762dea1-a7d1-4f2e-98e8-1bcd71fb7ebb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.598611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b09b8-2572-432f-b619-808d6cce0769 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.604214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.615551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.626927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.627531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.627808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.628079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.634304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.634897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.635172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.635440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.636120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:42.636422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.637857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.649559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 328/663] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:42.650071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.700122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.701880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.744829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f814abc0-d726-4025-b077-467aecb3418e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.778092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.778688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.778976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.779246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.804255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 336/664] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:42.805136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.818918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.823906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:42.848595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.850078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.902843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0912b9f-bb8c-40e3-9ad3-4741d1be31e4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:42.935693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:42.936281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:42.936551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:42.936846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:42.937526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:42.937823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:42.939248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:42.944493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 329/665] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:42.945346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:42.956267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 337/666] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:42.956267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.023714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.024245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.046705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62708192-aa96-4f90-9c42-49218e7a3845 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.052272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e231324-267e-4869-b4e6-9e0d717da052 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.081726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.082349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.082629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.082903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.085114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.085983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.086259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.086528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.087242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec86be5855254c8abc10fb636add2749) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:43.087544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.089020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.092731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:43.094779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 338/667] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/role_assignments?user.id=ec86be5855254c8abc10fb636add2749&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:43.095553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:43.098127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:43.099077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:43.099873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 330/668] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:43.100545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.216502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc51d0-16b3-44a5-8787-3506b2d22102 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.233522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66053f42-0574-4a79-9f54-5792ddfa6e64 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.255396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.256366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.256645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.256954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.260723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.270136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.272473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.276608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.277947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.277947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.277947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.281008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:43.287956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:43.289119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:43.290022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:43.290022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:43.290022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:43.290022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 339/669] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:43.290022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.295032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.296503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:43.314405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:43.315386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:43.316401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:43.316401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:43.316401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:43.316401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 331/670] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:43.316401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.408565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b60d78-c290-4dc1-bbcd-806fb59bc4b2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.443450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.443901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275607d-4062-45da-a069-f33b047ae346 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.444091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.444324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.444593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.445356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=3f6f37bc49c64ed1a9a61accef657525) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:43.445659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.447089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.452985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 340/671] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/role_assignments?user.id=3f6f37bc49c64ed1a9a61accef657525&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:43.453761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.498077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.499635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.501347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.501852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.502699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:43.503268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.505034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.518123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 332/672] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:43.519847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.527024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.585962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d69c4a-e765-4a96-a737-b7e1f7c7bf60 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.600132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.626747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.627501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.627852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.628195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.629283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:43.629708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.631472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.645341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 341/673] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:43.646403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.673205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40408416-9d3f-4d99-95da-c67c93c76a02 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.712630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.713274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.713554 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.713834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.714481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:43.714784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.716185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.729328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 333/674] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:43.730271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.775805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb583a50-d838-444d-a1c5-459135ea0e69 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.800047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.812404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.813071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.813359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.813734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.825508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.827198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.837127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a4e9-a44a-43e9-9f5d-f5d35c6b99d1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:43.849629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:43.850346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:43.851425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:43.852457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:43.852457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:43.852457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:43.852457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 342/675] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:43.853396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.880554 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:43.881316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:43.881666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:43.882004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:43.889164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:43.890692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:43.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:43.904182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 334/676] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:43.905185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:43.971135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9af91-b171-4d64-bf98-207b1a40460c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.005907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.006628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.006973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.007315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.014061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.015583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.025109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fbcb7-20a6-495e-9bfe-cfaa2b37df35 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:44.027760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:44.028803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:44.029883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:44.031072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 343/677] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:44.031937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.057185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:44.064689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.065349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.065633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.066432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.067202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:44.067522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.069170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.084480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 335/678] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:44.085396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.120813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:44.149816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46088434-833f-4dfd-8866-a68cd277a1d1 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.200223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.200856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.201371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.201657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.202432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:44.202705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.204195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.206079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa05666-5121-46bc-ba52-354de5511aba None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.222959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 344/679] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:44.223849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.247809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.248740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.249177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.249743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.250906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:44.251609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.253896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.268930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 336/680] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:44.269988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.313044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:44.332381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1ebf7-cf28-4a7e-b84a-793aec4eb065 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.368575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.369412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.369746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.370117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.380151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.381872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.395888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:44.398378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:44.399360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:44.400269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:44.408701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 345/681] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:44.408701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.411149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aaba2-3646-4bad-b6fa-e1efdc500fb4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.458797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.459463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.459744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.460210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.461263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec498f2822ba4dc9b05a0866e885b5b8) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:44.461574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.462995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.471102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 337/682] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/role_assignments?user.id=ec498f2822ba4dc9b05a0866e885b5b8&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:44.472159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.527348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea43d0-4ca2-4e85-8965-2d6d1acfe51b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.570966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.571489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.571764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.572030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.572792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=82ecac0d9cc54fefbca5e3b6d81c31b9) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:44.573094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.574894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.583012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 346/683] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/role_assignments?user.id=82ecac0d9cc54fefbca5e3b6d81c31b9&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:44.670920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150da44-a531-4ef5-980c-bc01aa065465 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.725584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.726216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.726500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.726775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:44.727530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:44.727826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:44.729404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:44.757274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 338/684] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:44.762862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:44.791790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:44.896944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564f71-bac1-4861-a5cb-e32e9a3b64d8 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:44.936374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:44.937383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:44.937861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:44.938330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.013071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.014898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.026132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:45.126366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 347/685] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:45.131640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.148058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:45.248212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01903200-4ac0-4ece-9154-2ed5f8b82e0e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.296017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.296648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.296958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.297228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.311016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.312477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:45.333783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:45.334737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:45.335603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:45.335603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:45.335603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:45.335603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 339/686] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:45.340524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.474970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f3e0d-abb8-4c3a-b916-cc086d1425a4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.489775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:45.521998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.522612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.522892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.523168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.523855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:45.524125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.525531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558e41d-1b5b-4e49-93f6-e4006f37b31e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.542009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 348/687] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:45.547100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.639745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:45.653980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2ae4-c2d8-4888-8964-e10aae135b33 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.654349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2ae4-c2d8-4888-8964-e10aae135b33 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.654701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd2ae4-c2d8-4888-8964-e10aae135b33 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.656342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 349/688] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:45.666221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f5ea9-da6b-47c9-b689-6eab863bd54d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.666582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f5ea9-da6b-47c9-b689-6eab863bd54d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.666935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f5ea9-da6b-47c9-b689-6eab863bd54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.692842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58549ed-2529-4e9c-8003-c94816a184f4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.699437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-585f5ea9-da6b-47c9-b689-6eab863bd54d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:45.702043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-585f5ea9-da6b-47c9-b689-6eab863bd54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:45.758212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35621947-8632-4931-b409-a9962a008e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:45.759155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621947-8632-4931-b409-a9962a008e89 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.759606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621947-8632-4931-b409-a9962a008e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.760068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35621947-8632-4931-b409-a9962a008e89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.772609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35621947-8632-4931-b409-a9962a008e89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:45.775768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35621947-8632-4931-b409-a9962a008e89 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-35621947-8632-4931-b409-a9962a008e89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:45.798982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:45.799924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:45.801344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:45.807199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 340/689] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:45.811461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35621947-8632-4931-b409-a9962a008e89 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:45.840877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 350/690] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:45.944044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a81d10-3f65-4bf7-b2a8-e27249495f7e None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.944666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a81d10-3f65-4bf7-b2a8-e27249495f7e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.944888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35621947-8632-4931-b409-a9962a008e89 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:45.945393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a81d10-3f65-4bf7-b2a8-e27249495f7e None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.948279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 351/691] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:45.952656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:45.952971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:45.953247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:45.986728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:45.990080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:45.992865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:46.009126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.010096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.011641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.012143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.013191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:46.013690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:46.015295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:46.029076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 341/692] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:46.036827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.145988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9edd3a-9aba-42f8-8e58-5b6a35614946 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.158663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 352/693] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:46.193221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.194177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.194668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.195131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.196041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=67e76f823b3a48368f5476c36acb9341) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:46.196556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:46.198313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:46.205186 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 342/694] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/role_assignments?user.id=67e76f823b3a48368f5476c36acb9341&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:46.228329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.276035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:46.386527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af31c7-932c-4a8d-b202-50482b244ccc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.437637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.438284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.438559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.438831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.460373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:46.463354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:46.465362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:46.466274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:46.467526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 353/695] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:46.474083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.528023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:46.590485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c32eb-4a8b-451d-a141-f007596a5489 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.616541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:46.629569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.630217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.630496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.630772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.631362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:46.632753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:46.646282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662239e-cab7-4987-b666-f563707745e7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:46.649220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:46.650221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:46.651051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:46.652318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 343/696] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:46.658411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.692445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.693210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.693586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.693927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.694605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:46.694905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:46.696478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:46.708163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 354/697] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:46.714416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.821007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ddb8e-682e-4a7a-be5a-7977a8b7e5cb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.847806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:46.890738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.891351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.891629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.891901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:46.892539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:46.892926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:46.895437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:46.905338 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 344/698] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:46.908994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f12203-2d20-497d-8a3d-da5b6c37608e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:46.911771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:46.984574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:46.985234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:46.985509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:46.985778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.006870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.009881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:47.032942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:47.035286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:47.037501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:47.037501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:47.037501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:47.039346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 355/699] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:47.044744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.047734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.068418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b41b1b-ac75-4019-a96e-a90bdf4f59b6 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.115712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.116320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.116602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.117143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.117788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:47.118090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.120383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9059c2e-8550-497b-a00f-26dc4f9be79b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.133905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 345/700] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 17 20:51:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:47.171075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c571a4-97c8-480c-a553-c35739be61ae None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.171075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c571a4-97c8-480c-a553-c35739be61ae None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.171075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c571a4-97c8-480c-a553-c35739be61ae None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.171075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 346/701] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:47.180419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.180787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.181003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.201676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:47.202906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:47.235718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cc255-2d9c-4715-821e-1b42892b0342 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.244147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.281253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 347/702] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:47.302621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.303377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.303756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.304133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.304938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:47.305395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.306938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.321932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 356/703] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:47.329139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.456111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686947f5-c7e5-4468-93e8-736bb697ea61 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.494640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.495286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.495569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.495842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.506302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.507808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:47.521359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:47.522342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:47.523948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 348/704] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:47.528449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.574694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.604043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.617983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.650342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ac9cc-07d7-43a0-91c3-f5c14722e4c9 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.688049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.688648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.689040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.689313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.690157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:47.690461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.692102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d6d24-4067-4fe2-b3a9-ad62476df5dc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.699139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bce91-e358-4110-8ab4-1c4e2060cb2b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.711437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 357/705] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:47.712704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817cce-f31f-420f-ae3f-acf33cf7c2ff None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.717538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817cce-f31f-420f-ae3f-acf33cf7c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.717827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817cce-f31f-420f-ae3f-acf33cf7c2ff None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.719342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 358/706] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:47.720099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817cce-f31f-420f-ae3f-acf33cf7c2ff None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.735668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.736295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.736574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.736880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] PATH_INFO: `/v3/services/network` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.738084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:47.739645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-7146eafc-7a6e-40e7-853c-e51b09da833f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:47.749146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:47.751413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:47.753440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 349/707] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:47.755288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.755288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.755288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.771318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:47.772602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:47.837818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817cce-f31f-420f-ae3f-acf33cf7c2ff None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.845258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 350/708] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:47.846237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:47.860723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.875963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:47.885965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.886617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.886894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.887166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.959107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03181bac-5c2b-4dbb-beac-c278f1d460a2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:47.997230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:47.997962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:47.998250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:47.998522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:47.999167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] RBAC: Adding query filter params (name=network) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:47.999469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.001199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.006369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 351/709] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:48.012359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.043164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.044637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.132433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:48.147047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6121600-8dc6-4cc1-b676-362c0c5b8255 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.188110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.188847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.189161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.189438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.190072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:48.190405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.191817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.196863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 352/710] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:48.197664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.311511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64740b30-0e3f-4920-ad47-789d82b0d0a1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.347702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 359/711] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:48.351513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.352349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.352880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.353304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.353984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.366128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.367859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:48.400730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:48.401940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:48.403034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:48.403034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:48.403034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:48.403034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 353/712] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:48.403034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.407394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:48.415966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:48.552407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138fcff-5c71-4421-97dd-70c4608d178b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.594116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.594666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.594945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.595213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.606961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.608480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:48.624998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:48.626033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:48.626947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:48.626947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:48.626947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:48.627158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 360/713] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:48.630121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cbd5e-9839-4138-b3e0-d9c0041e3d20 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.636707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.715546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.716169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.716450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.716750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.717402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:48.717584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:48.717685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.719072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.747335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 354/714] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:48.755029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.765221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a851-9523-4c62-a2f9-ef1f08232987 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.844263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.844834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.845113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.845387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.846033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:48.846333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.847785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:48.867428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:48.879865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 361/715] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:48.894145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:48.928762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6345b-4253-4c78-afb5-1a157b727d48 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:48.971073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bba084-a486-4522-8c0d-ab02d40378ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:48.971929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba084-a486-4522-8c0d-ab02d40378ae None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:48.972222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba084-a486-4522-8c0d-ab02d40378ae None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:48.972566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bba084-a486-4522-8c0d-ab02d40378ae None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:48.973272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bba084-a486-4522-8c0d-ab02d40378ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:48.974731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bba084-a486-4522-8c0d-ab02d40378ae None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.000493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 355/716] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:51:48 2022] POST /identity/v3/users => generated 254 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:51:49.076226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5915cf-6879-493b-abc8-eca3cdb738b8 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.111659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:49.118389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.119085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.119366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.119634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.127432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.129058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.134155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419e73c-4a97-4ad6-80ba-54aec9d22dfa None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.134764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419e73c-4a97-4ad6-80ba-54aec9d22dfa None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.135248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419e73c-4a97-4ad6-80ba-54aec9d22dfa None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.136817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 356/717] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-0114cd51-9d02-41e9-86df-10dae37769e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:49.147430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:49.148424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:49.149964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 362/718] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:49.155070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419e73c-4a97-4ad6-80ba-54aec9d22dfa None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.155851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.156117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.156391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.178619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:49.180388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:49.267761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 363/719] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:49.354710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419e73c-4a97-4ad6-80ba-54aec9d22dfa None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.396295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:49.413104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.414182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.414516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.414796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.415499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:49.415805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.417294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.434532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 357/720] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:49.440068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.569981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fa991-d23b-4730-aafb-3952ad67e177 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.579739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.608704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:49.611829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.612486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.612973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.613282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.614013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec86be5855254c8abc10fb636add2749) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:49.614322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:49.616644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:49.630295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 364/721] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:49 2022] GET /identity/v3/role_assignments?user.id=ec86be5855254c8abc10fb636add2749&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:49.728586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03de9f-abc1-4558-b4c1-e6d7c8a38420 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.779539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c53909-d6b1-41ce-81c4-9f5ce5bb43a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:49.780194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c53909-d6b1-41ce-81c4-9f5ce5bb43a8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.780478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c53909-d6b1-41ce-81c4-9f5ce5bb43a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.780784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c53909-d6b1-41ce-81c4-9f5ce5bb43a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.801747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c53909-d6b1-41ce-81c4-9f5ce5bb43a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-11c53909-d6b1-41ce-81c4-9f5ce5bb43a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:49.806690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:49.807658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:49.809272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 358/722] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:49.813764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:49.963486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fde0a5-cc82-4bf0-871e-0461ee6645f2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:49.981120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b985ef-4606-46d2-8068-f5726395f946 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.981619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b985ef-4606-46d2-8068-f5726395f946 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.981619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b985ef-4606-46d2-8068-f5726395f946 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:49.983180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 359/723] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:49.988647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:49.988969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:49.989245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.008417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:50.009682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.018253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.020038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.020579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.021058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.021908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:50.022400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.023960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.039808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 365/724] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:50.049007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.053042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.085799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 360/725] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 17 20:51:50.150564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.230929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1114dfc-77e5-4105-9a56-0d9b53740653 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:50.271047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.271661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.271942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.272218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.283527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.285037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:50.299251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:50.300803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:50.301683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:50.301683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:50.301683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:50.301850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 366/726] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:50.307508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.411162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae43c9-8dc2-40e7-aada-1578b424f24e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:50.421333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.480189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.482163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.483298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.484133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.485139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:50.486830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.491913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.519139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 361/727] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:50.529956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.581066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.633005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.638976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11575307-ee4b-47d0-a325-63d2acbf3ec7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:50.689761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0833-31dd-415d-9232-4ef39c496b72 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:50.732828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf34b9-5c0f-41cf-b62c-b8cc793ff2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.733450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf34b9-5c0f-41cf-b62c-b8cc793ff2f2 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.733729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf34b9-5c0f-41cf-b62c-b8cc793ff2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.734005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf34b9-5c0f-41cf-b62c-b8cc793ff2f2 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.734680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf34b9-5c0f-41cf-b62c-b8cc793ff2f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.736067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf34b9-5c0f-41cf-b62c-b8cc793ff2f2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.740972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8298e6-f661-4854-aa64-9240545472db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:50.741612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8298e6-f661-4854-aa64-9240545472db None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.741892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8298e6-f661-4854-aa64-9240545472db None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.742166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8298e6-f661-4854-aa64-9240545472db None admin] PATH_INFO: `/v3/users/swift` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.750241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8298e6-f661-4854-aa64-9240545472db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:50.754061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8298e6-f661-4854-aa64-9240545472db None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-be8298e6-f661-4854-aa64-9240545472db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:50.773208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:50.774615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:50.775618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 362/728] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:50.780347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8298e6-f661-4854-aa64-9240545472db None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:50.785557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 367/729] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:51:50 2022] POST /identity/v3/services => generated 251 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:50.815860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645786e3-06ab-4c82-9be4-27cc49ca99fe None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.816215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645786e3-06ab-4c82-9be4-27cc49ca99fe None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.816460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645786e3-06ab-4c82-9be4-27cc49ca99fe None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.818282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 368/730] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:50.826269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:50.826778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:50.827187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:50.864806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:50.867715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:50.956364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:50.974866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8298e6-f661-4854-aa64-9240545472db None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.018470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 369/731] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 17 20:51:51.022218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.022840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.023122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.023399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.024136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:51.024439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.025904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.052133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 363/732] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 17 20:51:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:51.058835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:51.146502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:51.169820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe60cbc-a177-4a2a-a589-5227cf66179b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.214834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.215438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.215715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.215985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.285837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.287300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.372453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:51.393130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 370/733] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:51 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 17 20:51:51.397794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:51.436757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:51.575793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702b06d-a56c-4d1c-a451-c853849416ba None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.605596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940a56-80cc-4f2d-a7e5-8622624ba3bb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.625237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.625805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.626092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.626371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.637622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.639072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:51.654611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:51.655585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:51.656446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:51.656446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:51.656446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:51.656446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 364/734] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:51.657506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:51.661611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:51.686029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.686594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.686872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.687144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.688017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.689228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.725965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 371/735] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:51:51 2022] POST /identity/v3/services => generated 240 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:51:51.802940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966384-6448-46e3-bf0f-2a7da6602591 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:51.865538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:51.866532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.867051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.867770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.868656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:51.869236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:51.871914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f8626-fd1d-4e49-bd64-d3181e83e820 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:51.889515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 365/736] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:51.896721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:51.981021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:51.987110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26b63-6cab-4437-b23f-8a9dcad2120a None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.987948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26b63-6cab-4437-b23f-8a9dcad2120a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.988284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec26b63-6cab-4437-b23f-8a9dcad2120a None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:51.994396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 366/737] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:51:51.997932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:51.998416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:51.998698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.020291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:52.021600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:52.071315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69c66c-ee7e-4047-8e44-497d982d05bc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.090512 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:52.116715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 367/738] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:51:52.126141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.126936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.127268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.127484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.134999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.136498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:52.149149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:52.150018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:52.150939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 372/739] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:52.155971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.283738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a4f1a-c110-4895-a861-360ce83f5835 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.325279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.325916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.326196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.326477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.327193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:52.327502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.329010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.342841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 368/740] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:52.349051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.469646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.497499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae0fb2-4da4-4149-b43d-e5672a483a26 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.515409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:52.549810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.550961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.551537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.551642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:52.552102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.553063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=67e76f823b3a48368f5476c36acb9341) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:52.553589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.555385 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793166d-571a-4d7b-a809-1ea9fa8ce058 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.562127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 373/741] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:52 2022] GET /identity/v3/role_assignments?user.id=67e76f823b3a48368f5476c36acb9341&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:52.563675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5873a-3731-49c4-b042-587ae01e63b6 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.564194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5873a-3731-49c4-b042-587ae01e63b6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.564709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5873a-3731-49c4-b042-587ae01e63b6 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.570696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 374/742] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:52.572453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.572988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.573469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.601769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:52.603376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:52.636263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cf8b6-51fb-4dea-90fe-bc21fede8446 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.705148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.706732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.707245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.707855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.734457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:52.742805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:52.743778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:52.744583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:52.747988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 369/743] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:52.754255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:52.762946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 375/744] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:52.877145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a48da-66c9-489e-b06c-5e3148489e02 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:52.885783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:52.923147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:52.923765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:52.924045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:52.924315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:52.925034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:52.925397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:52.926705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:52.938857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 370/745] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:52.945180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.085450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.088665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c2a92-c2fb-4052-8332-a6d8c344b564 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.106636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.136553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.137491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.137905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.138492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.158667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.160552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:53.188038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:53.189258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:53.190303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:53.190303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:53.190303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:53.190303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 376/746] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:53.196644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.255835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7afd4-32b8-4504-b96b-5fbf9f8302be None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.292219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.293513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.293969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.294385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.306250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:53.312161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:53.313326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:53.314335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:53.317671 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 371/747] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:53.331425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.421082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.428472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102dc7d-6cde-448d-b3f9-bd6dab3fc780 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.496467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae9870-8a9b-4d8a-9437-878df2f53dc1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.501723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.502597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.503844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.504318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.505207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:53.505805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.507418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.540246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 377/748] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:51:53.544144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.545481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.546061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.546351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.546616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.552537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:53.552871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.557365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb810539-b657-46cb-81c9-9bdf5dc31e30 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.564356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.569457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 372/749] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:53.570860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faa550-d437-4077-91ad-8f716f2dc4ab None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.571234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faa550-d437-4077-91ad-8f716f2dc4ab None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.571601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faa550-d437-4077-91ad-8f716f2dc4ab None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.572908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 373/750] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:53.577779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faa550-d437-4077-91ad-8f716f2dc4ab None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.662262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482e18a-3802-45b4-bb78-616d47ee6b7b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.670680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.683046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faa550-d437-4077-91ad-8f716f2dc4ab None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.703737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.705369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.705788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.706526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.715033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.716701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.723546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.724814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.725245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.725703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-35a0b79e-ea4b-4524-8921-b20717117140 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:53.729918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:53.731068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:51:53.732236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:53.734101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 378/751] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:53.735649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.736009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.736376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.744214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.746067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.756331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:53.757620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:53.764305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:53.765435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:53.766510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:53.766510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:53.766510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:53.768801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 374/752] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:53.769997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.835182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 379/753] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:53.836383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:53.838755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.889435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a6b97-4fa7-4d8a-9a11-8b2a37ab1b8c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.930683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:53.931529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:53.931945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:53.932354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:53.933266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:53.933705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:53.935267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:53.950838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:53.956265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 375/754] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:53.966575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7c9c-45f6-44ba-8fb5-7cf87926672f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:53.967728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.016472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.017486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.017605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.017967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.019251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:54.019650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.021148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.033257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 380/755] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:54.039474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.090065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29050d86-aa5f-4653-9d90-ac37a922dc7a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.131448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.132505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.133597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.134322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.134743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.211597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519521b-2425-49cf-97eb-6d0be82d150c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.213000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.216223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.218652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.295548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.296087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.296476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.296899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.305968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.307569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:54.319707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:54.320866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:51:54.321983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:54.328211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 381/756] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:54.329162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.339768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 376/757] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:51:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:51:54.340823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.434248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be308a6f-31f5-4081-a06a-2c130a0a9a76 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.457533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.457763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.461737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9394f0-53c7-43f3-b2e1-155ef6da6d00 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.472508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.473460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.473742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.474013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.474596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.475954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:54.486745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:54.487764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:54.488589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:54.489340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 382/758] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:54.490251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.501493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.502098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.502373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.502642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.504150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:54.504449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.506016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c2611-8078-4b73-a630-083d9a4364b6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.525120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 377/759] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:54.526200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e69e8-7a8b-4d6f-be7d-46ea8592ef06 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.526477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e69e8-7a8b-4d6f-be7d-46ea8592ef06 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.526745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e69e8-7a8b-4d6f-be7d-46ea8592ef06 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.531995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 378/760] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:54.531995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e69e8-7a8b-4d6f-be7d-46ea8592ef06 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.601061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8222-df26-48d9-96c8-a580b175845d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.634818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.635045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e69e8-7a8b-4d6f-be7d-46ea8592ef06 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.635442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.635718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.635984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.646913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.648384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:54.662624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:54.663729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:54.664601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:54.664601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:54.664601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:54.664601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 383/761] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:54.664853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.672576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.673205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.673479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.673748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.674373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:54.674667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.676267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173535b-6c5c-4bbd-ae70-c3d02fead32a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.681052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 379/762] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:54.681052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.681052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.681052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.701437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:54.702839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:54.714928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.722827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.776588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e29a-8a67-48ad-8f17-7a07d2c45fc4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.784849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 380/763] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:54.784849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.824383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.825147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.825489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.825870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.840438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.842801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:54.862871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:54.864053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:54.865157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:54.865157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:54.865157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:54.865157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 384/764] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:54.866456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.892582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440b6ec-6ebb-432c-87d7-c81396dcc497 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:54.938125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:54.938737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:54.939096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:54.939583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:54.941203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:54.941505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:54.943559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:54.959665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 381/765] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:54.960594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:54.978145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.980080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:54.988847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839ae13-01a3-4add-84a4-a2dbc937fc88 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.028183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.029017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.029388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.029754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.030529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.031016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.032640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.039284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 385/766] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.040183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.088705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819c1d3-d7ec-481e-9707-b24ca365ea37 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.129576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.130552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.131136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.131705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.132997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.133513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.135979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.152528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 382/767] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.152528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.170394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe24f6-ad3f-4146-b7f1-b197ed2820d4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.208688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.209305 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.209668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.209842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.216327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.217805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.224146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:55.229219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:55.230162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:55.231058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 386/768] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:55.231542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.247356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.258094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab4727-b95e-4897-b5fd-b7ed4820cb8c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.291268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.291879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.292161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.292432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.293178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] RBAC: Adding query filter params (interface=public, service_id=759bff57d34b48f19bba284be56d591d, region_id=RegionOne) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.293485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.294845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.303106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 383/769] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/endpoints?service_id=759bff57d34b48f19bba284be56d591d&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.303901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.333441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c770ad2-10b3-4c14-bcb5-6261f5198e6b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.366487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.367104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.367378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.367650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] PATH_INFO: `/v3/services/network` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.368218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.369604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a6e211cf-9b36-4dd8-bc2d-40332bf30327 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:55.378434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:55.379521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:55.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 387/770] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/services/network => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:55.381171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eb5c8-d675-4c1a-8c72-4a7d7ade4528 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.381171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eb5c8-d675-4c1a-8c72-4a7d7ade4528 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.381371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95eb5c8-d675-4c1a-8c72-4a7d7ade4528 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.382587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 388/771] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:55.383268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95eb5c8-d675-4c1a-8c72-4a7d7ade4528 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.408655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96370e-70fe-4a50-b7b5-b753389c5c42 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.443388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.444011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.444283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.444563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.447662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.451731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.453227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.464274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:55.466725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:55.467572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:55.468468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 384/772] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:55.468992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.491240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95eb5c8-d675-4c1a-8c72-4a7d7ade4528 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.524911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.525508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.525786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.526061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.526748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.527049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.528476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.541022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 389/773] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.541811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.566309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeede0b-3054-4716-9d74-1af4adebb150 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.599823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03835601-77a9-474c-b782-e751c306bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.600416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03835601-77a9-474c-b782-e751c306bf56 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.600714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03835601-77a9-474c-b782-e751c306bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.600993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03835601-77a9-474c-b782-e751c306bf56 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.601586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03835601-77a9-474c-b782-e751c306bf56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.602936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03835601-77a9-474c-b782-e751c306bf56 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.607846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 385/774] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/services => generated 772 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.609079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.609360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.609631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.629201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:55.630418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:55.662808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965713-b945-4ba2-9b75-aa89a4430dd2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.695898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.704084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.704990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.705274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.705550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.706723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] RBAC: Adding query filter params (name=network) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.707036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.708430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.712228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 386/775] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:55.713403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 390/776] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.713596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.714224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.717061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.841998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fc6ed-c80b-4bbd-ba46-7fd4e13f0baa None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.847610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e5288-8b13-4607-bd27-fa538a3d7001 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:55.883129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.883984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.884696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.884988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.885997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=6a042e5b3cc04a83a52b61b1e0bbac33) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.886312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.887754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.889512 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:55.890196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:55.890618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:55.890961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:55.891709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:55.892095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:55.894187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:55.894334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 391/777] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/role_assignments?user.id=6a042e5b3cc04a83a52b61b1e0bbac33&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.895435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.908251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 387/778] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:55.917774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:55.989162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:55.996965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:56.060577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0bc16-4630-446b-92f5-f5a8b2729dcf None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.087200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d91ee3-20f7-4070-86de-6d753f90c702 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.098658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.099191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.099460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.099731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.100452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] RBAC: Adding query filter params (type=network) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:56.100789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.102204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.108718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 392/779] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:51:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:56.109103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.139171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.140216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.141550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.141550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec86be5855254c8abc10fb636add2749/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.143500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=ec86be5855254c8abc10fb636add2749, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.145606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.161608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 388/780] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:55 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec86be5855254c8abc10fb636add2749/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:56.162675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.232465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a048ae-925c-43c1-8b4a-0a8631176239 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.287375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:56.296123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23090759-3d50-445b-b7e5-84834dd4b525 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.314454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.315108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.316102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.316440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.321396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:56.341705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:56.352406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:56.353406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:56.354204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 393/781] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:56.360206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.374525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.375146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.375420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.375686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.376393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] RBAC: Adding query filter params (interface=public, service_id=77547cb27dd9461aba237044a8076210, region_id=RegionOne) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:56.376712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.379130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.385117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 389/782] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/endpoints?service_id=77547cb27dd9461aba237044a8076210&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:56.392369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.493757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33035d5-a7bb-44e5-884d-90afad7a65ac None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.501048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada176d4-8ccb-40e3-b53c-62bcab215991 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.531593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.532200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.532525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.532831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.533498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:56.533798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.535236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.543928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.544335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 394/783] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:56.544647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.544946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.545216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.545845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.547194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.550351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.562042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 390/784] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:56.609919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:56.662466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd475c18-8fd8-40b6-be14-98cfad7e54b6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.701357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.702401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.702913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.703432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.715992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.717919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:56.738466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:56.739476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:56.740330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:56.740330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:56.740330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:56.742527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 395/785] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:56.749829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:56.821592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:56.884750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5361-0d77-4daa-aa44-c579afe8ceae None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:56.920922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37960-9287-41f0-b763-32f996164cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:56.921530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37960-9287-41f0-b763-32f996164cea None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:56.921809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37960-9287-41f0-b763-32f996164cea None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:56.922085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e37960-9287-41f0-b763-32f996164cea None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:56.922730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e37960-9287-41f0-b763-32f996164cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:56.923030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e37960-9287-41f0-b763-32f996164cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:56.924458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e37960-9287-41f0-b763-32f996164cea None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:56.940837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 391/786] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:56.943063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.069339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac56b6-8722-462d-9cde-30b3a31d5b0a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.073304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:57.111630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.112988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.113378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.113654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.125789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.127310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:51:57.140172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:57.141108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:57.142029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 396/787] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:57.147302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37960-9287-41f0-b763-32f996164cea None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.256841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:57.265066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e37960-9287-41f0-b763-32f996164cea None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.310628 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.311245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.311523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.311794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.312531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:57.312862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.314391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.340074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 392/788] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:57.345870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.462546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e01bc2-690b-4145-a152-d77bfc9256bc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.498122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.498744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.499072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.499269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.510361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.511908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:57.526254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:57.527171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:57.528050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:57.528050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:57.528050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:57.530891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 397/789] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:57.533439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.640426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07602c7d-4cf6-436f-8673-e82adfbca24f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.690778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.691315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.691591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.691863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.692563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:57.692886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.694303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.711748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 393/790] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:57.717403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.747221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:57.847566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0be4-1aae-4b50-bc1f-e8f34aa593c2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:57.881680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:57.882303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:57.882576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:57.882844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:57.889714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:57.891176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:51:57.903240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:57.904156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:51:57.905090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:57.908782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 398/791] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:57.911118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:57.929911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:58.045883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584c87-1301-4e44-a33e-7031b0c64acc None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.088191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.088821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.089101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.089371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.090059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:58.090355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.091765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a10a7ac-a728-474d-866d-406bdcfb49ab None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.112050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 394/792] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:51:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:58.118013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:58.249056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:58.287452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe01a8-2fc2-4e34-a339-261af85f9800 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:58.322710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9957bdb-cd13-452f-968d-b374a1aa3aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:58.323294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9957bdb-cd13-452f-968d-b374a1aa3aac None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.323568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9957bdb-cd13-452f-968d-b374a1aa3aac None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.323846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9957bdb-cd13-452f-968d-b374a1aa3aac None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/67e76f823b3a48368f5476c36acb9341/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.325822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9957bdb-cd13-452f-968d-b374a1aa3aac None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=67e76f823b3a48368f5476c36acb9341, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:58.327276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9957bdb-cd13-452f-968d-b374a1aa3aac None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:58.355115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 399/793] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:51:58 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/67e76f823b3a48368f5476c36acb9341/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:51:58.543447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec0304-0a3b-4188-9107-ca10ea5c02f9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.544509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec0304-0a3b-4188-9107-ca10ea5c02f9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.545033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec0304-0a3b-4188-9107-ca10ea5c02f9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.546888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 395/794] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:58.553623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585a978-12a8-4575-a494-51e2f78c18ff None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.554536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585a978-12a8-4575-a494-51e2f78c18ff None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.555031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585a978-12a8-4575-a494-51e2f78c18ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.574762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5585a978-12a8-4575-a494-51e2f78c18ff None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:58.576262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5585a978-12a8-4575-a494-51e2f78c18ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:58.677202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 400/795] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:58.794557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb786-47a8-4754-b36c-bb6b70a6a472 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.794865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb786-47a8-4754-b36c-bb6b70a6a472 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.795303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb786-47a8-4754-b36c-bb6b70a6a472 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.797153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 396/796] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:58.804050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddd2ad-2a10-4b18-8c5d-33f696b94c2c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.804367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddd2ad-2a10-4b18-8c5d-33f696b94c2c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.804607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddd2ad-2a10-4b18-8c5d-33f696b94c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.825535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0dddd2ad-2a10-4b18-8c5d-33f696b94c2c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:58.826647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0dddd2ad-2a10-4b18-8c5d-33f696b94c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:58.891574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:58.901398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 401/797] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:58.911568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fe3ae-0137-4a6a-a96b-6f17976ad994 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.911876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fe3ae-0137-4a6a-a96b-6f17976ad994 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.912122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fe3ae-0137-4a6a-a96b-6f17976ad994 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.913776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 397/798] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:58.918692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:58.918970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:58.919244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:58.944922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:58.946032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:59.008437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fe3ae-0137-4a6a-a96b-6f17976ad994 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.060663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 402/799] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:59.125382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:59.149217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fe3ae-0137-4a6a-a96b-6f17976ad994 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.184032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.184659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.184959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.185228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.195716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:59.200476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:59.201398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:59.202220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 398/800] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:59.209622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.219359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.324371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222a0d0-736c-4ca9-bb86-4e878aa4f444 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.362820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.363528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.363804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.364076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.364662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.366057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:59.376054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:59.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:59.378349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 399/801] 10.222.0.16 () {56 vars in 1129 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:51:59.386109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.434577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79111ee2-00d8-48e8-9acb-f10bdba6c0ed None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.470424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:59.481879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.482484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.482757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.483025 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.483693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:59.483989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.485406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.494051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 403/802] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:59.495004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.501194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d9e9-4490-474d-91bc-f4e7f4ba0df8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.538419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.539554 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.539834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.540104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.541208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:59.541525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.543583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfebf3-e580-4a31-9cbe-0f8d844ce5d7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.551609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 400/803] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:59.553239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69bd5d-cc48-4fa3-8000-bc6ec8c530c4 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.553590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69bd5d-cc48-4fa3-8000-bc6ec8c530c4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.553936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69bd5d-cc48-4fa3-8000-bc6ec8c530c4 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.555286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 401/804] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:51:59.556044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69bd5d-cc48-4fa3-8000-bc6ec8c530c4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.608392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f34b20-5244-47b1-b584-dce3ce29391a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.628941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:59.645149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.645781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.646059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.646334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.656516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:51:59.662162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:59.663100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:51:59.663911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 404/805] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:59.664573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.676216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69bd5d-cc48-4fa3-8000-bc6ec8c530c4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.717542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.718176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.718453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.718722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.726427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:59.731700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.733187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0055f77a-d0ba-40e8-9628-5cb4165a8c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:51:59.746933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:51:59.747893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:51:59.749309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:51:59.749309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:51:59.749309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:51:59.749309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 402/806] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:51:59.749309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5e45-093e-449e-82d9-ec020821037b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.749611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5e45-093e-449e-82d9-ec020821037b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.749680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5e45-093e-449e-82d9-ec020821037b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.769015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-864c5e45-093e-449e-82d9-ec020821037b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:51:59.770200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-864c5e45-093e-449e-82d9-ec020821037b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:51:59.788955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdafab6-d2bc-44e3-af9b-d0bd2ecbbc3c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.823688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:51:59.824273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:51:59.824546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:51:59.824839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:51:59.825560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:51:59.825855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:51:59.827332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:51:59.837896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 405/807] 10.222.0.16 () {56 vars in 1124 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:51:59.837896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.849774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 403/808] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:51:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:51:59.850563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5e45-093e-449e-82d9-ec020821037b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:51:59.907175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:59.968124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5e45-093e-449e-82d9-ec020821037b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:51:59.994562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a072-6d3f-4b07-ab5a-3d96e5ec8bb2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.010463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.010925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.011097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.011356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.011629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.012276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.012595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.013991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.032698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 404/809] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:00.032698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.039958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.040565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.041338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.041621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.042618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.042988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.045164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.057320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 406/810] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:51:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:00.058233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.145017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc1206-f2f8-45d7-be06-fd41e4e41736 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.153412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.175135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875fd49-aa82-403e-9da7-5241943fbcb0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.182032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.182635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.182914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.183190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.183844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.185232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.212153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 405/811] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:00.212988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.233483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.234128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.234404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.234732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.246796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.248722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:00.265287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:00.266421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:00.267352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 407/812] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:00.267936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.311069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.370999 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6f6d8-5fcc-4d23-b4fa-936e0992916f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.417225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.418685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.419072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.419447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.425580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85e9a6-ed01-41ef-bf28-b70f5cd6adde None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.432774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.434256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:00.449415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:00.450363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:00.451968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:00.451968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:00.451968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:00.451968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 406/813] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:00.451968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.468974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.469608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.469882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.470157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] PATH_INFO: `/v3/services/network` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.470746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.472686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.476385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:00.481995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:00.483004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:00.484293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 408/814] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:00.485225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.588035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b93a4c-94aa-4b1a-9760-594ea4b13b62 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.597996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.605663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a799c06-041e-4904-8903-58ac51ade94c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.624506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.625242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.625794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.625912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.626725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.627100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.629573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.645400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.646566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.647020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.647448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.648234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.648693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.653458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 407/815] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:00.653458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.655136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.670041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 409/816] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:00.671223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.726037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.758084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1d08b-bc60-4840-9879-53bc95c54863 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.784910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac6b96-81c0-46d4-82f4-ce35e3c152d5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.794720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.795337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.795935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.795935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.796557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:00.796883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.798246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.804004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 408/817] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:00.804867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.823606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.824467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.824919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.825336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.837070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.838740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:00.854818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:00.855886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:00.856787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:00.856787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:00.856787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:00.863325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 410/818] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:00.864431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:00.874465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.920834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80cd25-12c8-4f77-8d66-6e107d56a2d1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.957369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:00.958226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:00.958471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:00.958818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:00.961937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:00.968245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:00.970289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:00.982571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e48e9-9b8c-4e7f-a701-dff68c49a330 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:00.984953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:00.986049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:00.987330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:00.990067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 409/819] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:00.991079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.024727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.025335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.025663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.025940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.026573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] RBAC: Adding query filter params (type=network) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:01.026874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.028248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.038136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 411/820] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.038136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.105266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac54f86-1bad-4dc5-85c0-a2fc04c21c72 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.112645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.141886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.142487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.142761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.143028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.144210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:01.144502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.144896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2f40f-03c8-44fe-a43f-48012e950354 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.145897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2485772-a72d-476d-9e75-d8f3ac945127 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.159217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 410/821] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.160226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dff45-4a70-4de9-a40e-4a8ea7a6f7f7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.160508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dff45-4a70-4de9-a40e-4a8ea7a6f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.160808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dff45-4a70-4de9-a40e-4a8ea7a6f7f7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.162153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 411/822] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:01.166723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dff45-4a70-4de9-a40e-4a8ea7a6f7f7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.182740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.183338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.183614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.183928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.184662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:01.184994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.186426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.210558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 412/823] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.211245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.226625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.275187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dff45-4a70-4de9-a40e-4a8ea7a6f7f7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.311440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b9546-6951-4095-ab4b-25c447a78708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.312381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9546-6951-4095-ab4b-25c447a78708 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.312839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9546-6951-4095-ab4b-25c447a78708 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.313268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b9546-6951-4095-ab4b-25c447a78708 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.314164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b9546-6951-4095-ab4b-25c447a78708 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.316373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b9546-6951-4095-ab4b-25c447a78708 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.318121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36fce0-817b-4518-8f03-0099eb5dff15 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.338278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 412/824] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:01 2022] POST /identity/v3/endpoints => generated 337 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:01.340490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.340935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.341504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.345341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.362560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.363383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.363665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.363921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.372148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:01.373408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:01.378645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.380549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:01.402932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:01.403763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:01.404648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:01.407222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 413/825] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:01.408316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.464711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 413/826] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:01.464711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.516959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.552573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b98172-9e64-4a07-b70d-7edc14e365a2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.594253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e3000-3277-4f68-9851-962dd5e92f8f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.595079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.595686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.595966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.596236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.640940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.641580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.642343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.642619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.643459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:01.643930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.645417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.658410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.662734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 414/827] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:01.667947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.683065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.685044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.778810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677e52f-aca7-4e91-a86b-6c7e42330181 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.780407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.794903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 414/828] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:01.802070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.821538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.822135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.822419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.822685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/6a042e5b3cc04a83a52b61b1e0bbac33/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.838470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=6a042e5b3cc04a83a52b61b1e0bbac33, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:01.840013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:01.860715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 415/829] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:52:01 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/6a042e5b3cc04a83a52b61b1e0bbac33/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:01.860715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:01.896916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.926800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3ab76-be11-483f-90bb-57d90cc8afc6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:01.989007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:01.989762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:01.990107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:01.990454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:01.995442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089af51-320b-4b52-aa36-d2ed53a62f42 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.002148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.004225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:02.022696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:02.023694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:02.024556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:02.024556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:02.024556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:02.024556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 415/830] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:02.029573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.041393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.042606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.042978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.043322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.059733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:02.064720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:02.066750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:02.068385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 416/831] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:02.072109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.090203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.162039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e059a09-9e06-45d5-87a3-1465a1f91997 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.195251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8d246-9cfb-4eac-a63a-f0da46fb3bf1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.205309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.206513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.206939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.207668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.208820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:02.209284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.211460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.214987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.233609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.234395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.234769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.235186 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.235984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:02.236422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.238611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.255054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 417/832] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.257898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.343085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.368854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 416/833] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.369775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.372021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266d98f-0ce2-427c-aee9-7b1458030914 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.415791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.417137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.417600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.418049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.432470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.435033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:02.455145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:02.456666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:02.457594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:02.457594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:02.457594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:02.464896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 418/834] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:02.466658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.491941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88284d07-c7e1-4dc1-b733-710c7a314f37 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.541914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.543198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.543676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.544173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.553473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.555338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:02.570089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:02.571023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:02.571896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:02.574477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 417/835] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:02.580331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.595449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bc983-a19a-4d8a-877f-aeefaa9d71d1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.624110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.635556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.636167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.636445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.636750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.637511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:02.637769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.639064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b15f8e-7f21-4ac6-91bf-e876eb0664d1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.650790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.656594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 419/836] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.656594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35827007-c2da-4776-bf51-ff4b0abf5a81 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.656594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35827007-c2da-4776-bf51-ff4b0abf5a81 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.656594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35827007-c2da-4776-bf51-ff4b0abf5a81 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.656594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 420/837] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:02.661250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35827007-c2da-4776-bf51-ff4b0abf5a81 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.702857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12db1b-5b35-4c84-825d-1c741d8383cb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.738282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.738883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.739163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.739439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.740129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:02.740434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.741906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d06ee-01b7-45bb-bf46-e1ce687d83ad None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.754558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 418/838] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:02.756261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.756542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.756854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.771300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35827007-c2da-4776-bf51-ff4b0abf5a81 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.776246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:02.777453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:02.812167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:02.813300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:02.813782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:02.814301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:02.822445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:02.824140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:02.838112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:02.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:02.839973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 421/839] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:02.840563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.852035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 419/840] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:02.852768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:02.899161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.922003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:02.965244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddf801-b9ef-41ef-b2f5-caec77bb2c48 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:02.995923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c47ee-cfda-43a0-a991-0ba94727d600 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.000466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.001156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.001508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.001849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.003202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec86be5855254c8abc10fb636add2749) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:03.003654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.005510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.012259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 422/841] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/role_assignments?user.id=ec86be5855254c8abc10fb636add2749&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:03.047201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.047829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.048107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.048381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.049170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:03.050099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.051918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.074656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 420/842] 10.222.0.16 () {56 vars in 1135 bytes} [Sat Sep 17 20:52:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:03.080941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.152934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.214919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42bc14-f1a3-4a8b-bb31-75fbf9057478 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.226976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:03.260132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.260855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.261137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.261412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.279523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3292a3b2-f40a-47e4-9748-9f25022b9113 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.282307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:03.320264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.320921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.321205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.321524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.322107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.323688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:03.333932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:03.335123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:03.336138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 421/843] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:03.342343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.345241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.347419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.457315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e1fb0-096b-423c-8b34-2fdf31d3ac76 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.467013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 423/844] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:03.472885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.488929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:03.510544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.511295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.511919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.512287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.513515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:03.514179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.515644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.524717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 422/845] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:03.527687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.570046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:03.598465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea168b1-a5e7-41f4-8a94-65736a0e031b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.638280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.639267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.639613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.639956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.651285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.652956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.658120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee04f82-f6a4-45cb-a497-1a21b56b4ea2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:03.669229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:03.670198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:03.671071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:03.671071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:03.671071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:03.675379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 424/846] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:03.677013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.699041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.699930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.700385 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.700892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.701698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:03.702276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.703858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.716919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 423/847] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:03.750239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:03.796483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110e9f3-2a8b-431f-9283-56743595b3a7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:03.840383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:03.841040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:03.841636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:03.841907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:03.842569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:03.842869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:03.844248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:03.859353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 425/848] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:03.865174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:03.979596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:03.985809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d148f-b414-4e94-806d-64fd0e9f5b6a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.026929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.027561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.027834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.028105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.037211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:04.038707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-556978c1-1a67-4c56-9f93-dc531966efb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:04.051817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:04.052882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:04.054435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 424/849] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:04.059123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.127157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3698e6c-9865-4609-a10c-2d4c27bc630f None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.128025 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3698e6c-9865-4609-a10c-2d4c27bc630f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.128787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3698e6c-9865-4609-a10c-2d4c27bc630f None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.130567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 425/850] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:04.139354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.139991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.140464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.159738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:04.161378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:04.174913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d65f6-63f2-4496-9078-0af64f73f077 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.248779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 426/851] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:04.333191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:04.345405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.346169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.347460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.347750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.348454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:04.348778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:04.350325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:04.363245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 426/852] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:04.369142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.459957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:04.477583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2223036-380e-47b5-8348-cc5cf46e828e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.515476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.516039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.516315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.516583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.517343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=67e76f823b3a48368f5476c36acb9341) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:04.517643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:04.519102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:04.525402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 427/853] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/role_assignments?user.id=67e76f823b3a48368f5476c36acb9341&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:04.547257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.667414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91651df8-d09c-4917-8304-b91ece40aef9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.707663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.708288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.708561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.708861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.719581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:04.724391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:04.725424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:04.726535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 427/854] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:04.733626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.840106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942da66-fd08-4b5c-b816-1bdbf5540220 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:04.885673 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:04.886271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:04.886546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:04.886815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:04.887463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:04.887813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:04.889205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:04.897880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 428/855] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:04.904790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:04.954750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:05.013268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7144d2-6d14-4b58-8c2c-acf138277391 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:05.047889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:05.048929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.050046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.050580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.061144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:05.062822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:05.078251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:05.079491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:05.080408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:05.080408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:05.080408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:05.086416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 428/856] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:05.088937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:05.109400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:05.199712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d5285-7faf-4f7f-af16-0935e2d3d206 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:05.235171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:05.235811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.236071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.236337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.237778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:05.238092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:05.239599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4995a44-9feb-4396-a458-adab66b9b0a4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:05.253971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 429/857] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:05.259915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:05.356693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:05.359386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83868d-c8f9-4f17-9df0-99f85b77241c None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.359800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83868d-c8f9-4f17-9df0-99f85b77241c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.359920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83868d-c8f9-4f17-9df0-99f85b77241c None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.362061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 430/858] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:05.368384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ba282-8505-4655-85c4-366ab9f6ad49 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:05.368727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.369226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.369711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.400368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:05.401896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:05.408124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:05.409459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.409739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.410005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.420237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:05.421947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:05.438679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:05.439684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:05.440724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 429/859] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:05.449612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:05.492970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 431/860] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:05.563965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb302d-733f-4507-b734-c4687b180215 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:05.601362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:05.602387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.602941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.603420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.604351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:05.604930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:05.606534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:05.628863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 430/861] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:05.633768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:05.636533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:05.737914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477db43-f598-4c37-9380-4a04d05fa4b0 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:05.777498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:05.778099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.778375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.778647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.779865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:05.796113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:05.850709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:05.852350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:05.854318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:05.901005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e639bc-ea75-45a6-b079-2759d7e61837 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:05.947464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:05.948425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:05.948928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:05.949528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:05.950815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:05.952658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:05.962924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:05.963966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:05.964929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:05.967859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 431/862] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:05.976794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:05.993067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 432/863] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:05.999096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.068063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.095277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0454ae5-9af0-43f2-a338-9fd433e4a672 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.113809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500e014-cf15-42f4-b3f2-ee0d7d8c22c2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.115440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.132287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.132949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.133232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.133506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.134130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:06.134430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.135783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ee83-fc67-4090-8f53-3c4f24494093 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.146517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 432/864] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:06.148415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951292e-b2b6-47bc-981f-d9b3b9830508 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.148630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951292e-b2b6-47bc-981f-d9b3b9830508 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.149036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951292e-b2b6-47bc-981f-d9b3b9830508 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.152777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 433/865] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:06.152777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951292e-b2b6-47bc-981f-d9b3b9830508 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.154299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.155084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.155371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.155644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.166932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.168374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-1c4a3ec6-d7df-44f6-911d-5dc14a7c4dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:06.182957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:06.183910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:06.184826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:06.184826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:06.184826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:06.184826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 433/866] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:06.189999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.190364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.190644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.208647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:06.210032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:06.292710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 434/867] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:06.292710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.292994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951292e-b2b6-47bc-981f-d9b3b9830508 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.341113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.342212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.342490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.342758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.343798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:06.344370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.346108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.353082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 434/868] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:06.401051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.414328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb0948-b819-4e53-aafb-f3eb99e112ae None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.454653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.455272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.455628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.456477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.457473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:06.457779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.459191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.473200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 435/869] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:06.478634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.482595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.575815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.612111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b255b1a-e3ba-433b-9bb0-47848c394b7c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.658490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.660099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.660596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.661125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.671258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.673607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.685150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77136a4-accc-4c48-b483-f78e0b45ca05 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:06.691539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:06.692399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:06.693988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:06.700934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 435/870] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:06.702913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.725775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.726380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.726658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.726932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.727601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.729259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.753358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 436/871] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:06 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:06.781470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.830209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edc351-3446-482f-8ce8-7231d5f0fe2c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:06.883660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:06.884322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.884601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.884926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.885638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:06.885944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:06.887433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fcb53-6237-4d31-a187-d1a6fb053a36 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:06.904947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 436/872] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:06.908590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:06.912983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:06.922577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c10a5-c9df-4e4e-9e16-5cdd1e0bd23b None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.922858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c10a5-c9df-4e4e-9e16-5cdd1e0bd23b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.923071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c10a5-c9df-4e4e-9e16-5cdd1e0bd23b None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.925310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 437/873] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:06.931515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:06.931823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:06.932061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:06.952853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:06.954972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:07.037935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 438/874] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:07.044610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa109c9-4a88-4d29-bf24-2622106a7a78 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:07.081248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:07.081813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:07.082086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:07.082356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:07.083077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=6a042e5b3cc04a83a52b61b1e0bbac33) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:07.083377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:07.084950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:07.091221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 437/875] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:06 2022] GET /identity/v3/role_assignments?user.id=6a042e5b3cc04a83a52b61b1e0bbac33&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:07.317661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:07.371314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:07.424922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3afe6f-02b0-41b4-9b51-fbf83eb0f3e0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:07.460170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:07.460826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:07.461102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:07.461370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:07.461947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:07.463295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:07.474104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:07.475079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:07.476197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 439/876] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:07.482188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:07.555095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:07.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a2481-08ea-41b0-9c18-10add2a6a5ee None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:07.628314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:07.628946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:07.629222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:07.629492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:07.630118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:07.630415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:07.631781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf67e-f60d-4a98-8477-27c2341d9c5a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:07.637067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 438/877] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:07.642717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:07.748708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfcdd6-a804-412d-9182-786e121891d4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:07.762605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:07.785080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:07.785973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:07.786303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:07.786579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:07.787217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:07.787516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:07.788896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5e5f2-06d6-4f39-bd96-5552eda0e0d7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:07.793922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 440/878] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:07 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:08.848000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be60de-c3bc-43c4-ac07-a9ed84d16bb2 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.848784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be60de-c3bc-43c4-ac07-a9ed84d16bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.848784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be60de-c3bc-43c4-ac07-a9ed84d16bb2 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.850160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 439/879] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:08.858157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0cfc8-9299-4ade-8ab5-1b1a8dc9c4db None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:08.858872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0cfc8-9299-4ade-8ab5-1b1a8dc9c4db None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:08.859100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0cfc8-9299-4ade-8ab5-1b1a8dc9c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:08.888911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4dc0cfc8-9299-4ade-8ab5-1b1a8dc9c4db None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:08.890897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4dc0cfc8-9299-4ade-8ab5-1b1a8dc9c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:09.005049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 441/880] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:09.128728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64907-fed7-47f6-bf0a-587b2dd8239b None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.128937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64907-fed7-47f6-bf0a-587b2dd8239b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.129184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64907-fed7-47f6-bf0a-587b2dd8239b None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.130755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 440/881] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:09.136001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.136288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.136564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.155843 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:09.157027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:09.229687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 442/882] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:09.286700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64907-fed7-47f6-bf0a-587b2dd8239b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.398439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64907-fed7-47f6-bf0a-587b2dd8239b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.428874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:09.433623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.434227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.434817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.435090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.435758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.437207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.461188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 441/883] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:09 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:09.504336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.548424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:09.616362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f84f3-39fb-453b-995d-197621d46ea2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.664376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.664959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.665233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.665497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.666062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.667401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b5a98925-d44e-4af3-8059-95fe8a545bc2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:09.676400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:09.677423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:09.678235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 443/884] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:09.683701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.750227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b253ce-4200-4621-9601-263364619fd4 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.750508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b253ce-4200-4621-9601-263364619fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.750785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b253ce-4200-4621-9601-263364619fd4 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.753062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 444/885] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:09.760666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.760987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.761267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.779973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:09.781201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:09.794682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0ea73-ec3d-4a57-842d-774493ed2112 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:09.837632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:09.838242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:09.838519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:09.838792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:09.839423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:09.839718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:09.841154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:09.846095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 442/886] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:09.853610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:09.867278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 445/887] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:09 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:09.920652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:09.974117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acaf443-b9f5-40f5-a1e2-c7cb35016ebc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.010139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.011193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.011678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.012175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.013092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:10.013661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.015284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839da2b-0d9d-4f11-868b-897e342c56d8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.021208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 443/888] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:10.027272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.057552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:10.060948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d87eb-313c-4ad4-a2a9-6063339cc5b3 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.062028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d87eb-313c-4ad4-a2a9-6063339cc5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.062028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d87eb-313c-4ad4-a2a9-6063339cc5b3 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.063088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 444/889] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:10.068944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.069217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.069492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.087859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:10.089381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:10.148528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6378a0-8592-49f1-a102-7fbe3d474998 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.164584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 445/890] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:10.165492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.188377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.189335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.189677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.190329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.191109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] RBAC: Adding query filter params (interface=public, service_id=1997c8d7afc94219b66fd892408996bd, region_id=RegionOne) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:10.191483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.192997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.200823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 446/891] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/endpoints?service_id=1997c8d7afc94219b66fd892408996bd&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:10.206394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.233280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:10.288991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19fba5-3a77-41f3-a6c6-7a90c5153f9a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.324176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:10.328641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.329514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.329843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.330301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.337002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74d2f-d09c-49e8-a98b-5463ecc78da9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.342503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:10.348398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:10.349413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:10.355184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 446/892] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:10.358128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.387718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.388467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.388697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.388967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.392664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.394674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.404765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 447/893] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:10.436061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.520252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7853120-f861-40c8-bb77-0688f2ddd5bd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.536706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:10.564989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cd0b2-285f-4478-93a5-41b4b3d0ede0 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.565581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.566263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.566542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.566825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.567515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:10.567817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.570671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.584724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 447/894] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:10.588786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.620788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.621558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.621835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.622455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.623110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.624466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.646514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 448/895] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:10 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:10.707350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b200f0-0fb9-49aa-8c7b-e518089b7531 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.754040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.754685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.754962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.755234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.767761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.769891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:10.786687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:10.787666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:10.788548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:10.788548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:10.788548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:10.788548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 448/896] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:10.794226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.815971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:10.906247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfe444-dadc-4fe1-aa0d-f60395ef7780 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:10.940883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:10.941522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:10.941806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:10.942078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:10.942722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:10.943021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:10.944450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:10.957718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 449/897] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:10.963520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:10.986836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:11.069111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0815816-c713-4a0f-80ef-a59841dbf037 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.104889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.105520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.105800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.106075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.114032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.115484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:11.126973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:11.128972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:11.129915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 449/898] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:52:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:11.134739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.223086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:11.248473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15482282-89a6-4559-9007-ccfbea5d189a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.287582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.288553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.289110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.289584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.290546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:11.291050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.292745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.312406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 450/899] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 17 20:52:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:11.323633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.420302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:11.441944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4bed7-34c3-4326-a770-cba47574ec7e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.478175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b050e8-4962-47d9-83d2-94a05354b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.478778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b050e8-4962-47d9-83d2-94a05354b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.479054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b050e8-4962-47d9-83d2-94a05354b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.479329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b050e8-4962-47d9-83d2-94a05354b2d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.550955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b050e8-4962-47d9-83d2-94a05354b2d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.552429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b050e8-4962-47d9-83d2-94a05354b2d7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.641488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:11.663146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 450/900] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:11 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:11.669235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.748224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90c78-bec5-445e-92da-916b3d205f8d None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.748538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90c78-bec5-445e-92da-916b3d205f8d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.748822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90c78-bec5-445e-92da-916b3d205f8d None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.750418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 451/901] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:11.757572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.757974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.758818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.777425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9e693-7314-4e2e-b4bb-cd0000f06f00 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:11.783075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:11.784914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:11.818324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:11.818956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:11.819232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:11.819499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:11.822507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:11.833182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:11.835569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:11.854054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:11.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:11.855910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:11.855910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:11.855910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:11.856884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 451/902] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:11.860611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:11.884697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 452/903] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:11.967357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1a60b-6983-4717-b3a8-748eb3a48e73 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.001956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.002765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.003046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.003317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.003953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:12.004248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.005676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.017678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 452/904] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:12.026000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.135085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0befd2-b476-4ddc-872f-ec61d5093100 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.146708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.171190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.172255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.172753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.173271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.177276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.181526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.183242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:12.195714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:12.196592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:12.197515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:12.199892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 453/905] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:12.206755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.228759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.288439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa2d14-fe03-433f-bf80-274f0bee302c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.311013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff26cca-9c93-4e0b-9481-0e9138dc61bf None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.327686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.328557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.329089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.329746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.330678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.332175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:12.341694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:12.342672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:12.343518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:12.351525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.351525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.351525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.351525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.351888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:12.351888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.352146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 453/906] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:12.353320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.353572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.373430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 454/907] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:12.373430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.404784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.477661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b6b2e-0158-4468-ba57-d10bd3baa66f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.479077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab8858-2c1d-473b-a3d9-ecbd6bba7ded None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.491130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.513615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.514314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.514692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.515034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.516050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] RBAC: Adding query filter params (role.id=c722fc4c0c804be2a4eb9cadb419c454, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=6a042e5b3cc04a83a52b61b1e0bbac33) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:12.516425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.518658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.519458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.520262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.520841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.521280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.522169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:12.522635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.524154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.534907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 455/909] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/role_assignments?user.id=6a042e5b3cc04a83a52b61b1e0bbac33&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=c722fc4c0c804be2a4eb9cadb419c454 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:12.535265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 454/909] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:12.536325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.650030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:12.656833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545516e4-562b-4590-8b43-1850ce999378 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.696525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.697522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.697976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.698424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.699314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:12.699796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.701633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd720d0-8bff-4008-a6c4-48ee90f7068b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.708339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 456/910] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:12.717926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.728936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04445bd5-0fbe-45b6-a787-979f245877e2 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.729578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04445bd5-0fbe-45b6-a787-979f245877e2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.729830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04445bd5-0fbe-45b6-a787-979f245877e2 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.731243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 457/911] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:12.737465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.737833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.738108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.756275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:12.757485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:12.848881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e426a-0d25-49f3-b87a-7747f6a7f11b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:12.849074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 458/912] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:12.883105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:12.883713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:12.884042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:12.884319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:12.885069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] RBAC: Adding query filter params (interface=public, service_id=1909ff54eee84afbb8d3fb1faae6deff, region_id=RegionOne) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:12.885368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:12.886808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea542b3e-5ce9-4c53-880f-dbe902e51ff0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:12.892449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 455/913] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/endpoints?service_id=1909ff54eee84afbb8d3fb1faae6deff&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:12.898608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:12.993690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.007508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9e85-bec8-4f20-8509-d3412df7eba1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.044398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f7cb0-18b0-46f9-94b3-9c1bb8c14965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.045062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7cb0-18b0-46f9-94b3-9c1bb8c14965 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.045307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7cb0-18b0-46f9-94b3-9c1bb8c14965 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.045576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7cb0-18b0-46f9-94b3-9c1bb8c14965 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.046177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f7cb0-18b0-46f9-94b3-9c1bb8c14965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.047546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f7cb0-18b0-46f9-94b3-9c1bb8c14965 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.054517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 459/914] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:52:12 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:13.062300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89827609-837b-4c4d-94f1-f592f14f869f None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.062878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89827609-837b-4c4d-94f1-f592f14f869f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.062878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89827609-837b-4c4d-94f1-f592f14f869f None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.064473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 456/915] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:13.070765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.071058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.071394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.091855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:13.093136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:13.171403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 460/916] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:13.185104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89827609-837b-4c4d-94f1-f592f14f869f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.232690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.296901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89827609-837b-4c4d-94f1-f592f14f869f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.333457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.334129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.334421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.334695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.335334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.336710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:13.345985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:13.347017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:13.347847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 457/917] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:13.353851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.445297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.465632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.465966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cc0db-637d-4925-a681-1dc2e4a9742f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.504918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.505576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.505863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.506134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.506764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:13.507063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.508436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.518376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 461/918] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:13.519440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.593109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008b8ed-97c3-4295-98e4-ad43d142a766 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.594639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.632382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc47c8e-5f18-4ffc-99ca-bce3b24b82a6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.633176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.633808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.634085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.634358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.634939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.637502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:13.648163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:13.649226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:13.655235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 458/919] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:13.657982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.678682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.679303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.679563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.679831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.680573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:13.680920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.682286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.692792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 462/920] 10.222.0.16 () {56 vars in 1114 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:13.694745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.708604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.796216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdeb4a-9660-4146-90db-d19a80b58b86 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.818585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ad48c-8a07-4f8e-83ac-c6a4704db8b9 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:13.836474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.837952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.838700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.839456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.840921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:13.841738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.845013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.857627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 459/921] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:13.858847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:13.859834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:13.860508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:13.860850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:13.861122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:13.862468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:13.864122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:13.896707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 463/922] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:13.905423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:13.986319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd295bd-c5c9-4dd0-86f2-c90acc5e75a5 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:14.031881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a680fe9-1470-460d-8701-841638115081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:14.032820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680fe9-1470-460d-8701-841638115081 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.033289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680fe9-1470-460d-8701-841638115081 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.034030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680fe9-1470-460d-8701-841638115081 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.034903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a680fe9-1470-460d-8701-841638115081 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:14.035366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a680fe9-1470-460d-8701-841638115081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:14.036951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a680fe9-1470-460d-8701-841638115081 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:14.042908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 460/923] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:14.053880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:14.126604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:14.165843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a0b40-414f-4877-9ea9-8c51c5d645ab None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:14.200999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:14.201598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.201878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.202147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.202853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] RBAC: Adding query filter params (interface=public, service_id=b52980edcb644480a5a6c9cb021d2e9b, region_id=RegionOne) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:14.203159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:14.204815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73dde7e-e5fc-462a-8560-78fb2ef8cd7b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:14.213528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 464/924] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:52:14 2022] GET /identity/v3/endpoints?service_id=b52980edcb644480a5a6c9cb021d2e9b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:14.219271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a680fe9-1470-460d-8701-841638115081 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:14.310944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:14.321928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a680fe9-1470-460d-8701-841638115081 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:14.357660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105dcdd-20fe-4808-a0fc-f5446567b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:14.358329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105dcdd-20fe-4808-a0fc-f5446567b25d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.358610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105dcdd-20fe-4808-a0fc-f5446567b25d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.358880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105dcdd-20fe-4808-a0fc-f5446567b25d None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.359489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105dcdd-20fe-4808-a0fc-f5446567b25d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:14.360892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105dcdd-20fe-4808-a0fc-f5446567b25d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:14.372029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 461/925] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:52:14 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:14.776376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b7aa-954b-4449-81fd-6c0435a056d5 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.776742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b7aa-954b-4449-81fd-6c0435a056d5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.777029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b7aa-954b-4449-81fd-6c0435a056d5 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.778537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 465/926] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:14.784025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c490-72c3-414f-9386-da19925e661b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:14.784309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c490-72c3-414f-9386-da19925e661b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:14.784587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64c490-72c3-414f-9386-da19925e661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:14.803193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-db64c490-72c3-414f-9386-da19925e661b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:14.804737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-db64c490-72c3-414f-9386-da19925e661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:14.877774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 462/927] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:15.147258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b7aa-954b-4449-81fd-6c0435a056d5 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.258840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b7aa-954b-4449-81fd-6c0435a056d5 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.291699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:15.298688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815dbd06-334d-4aa9-b170-51dce55490d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.300071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dbd06-334d-4aa9-b170-51dce55490d6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.300419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dbd06-334d-4aa9-b170-51dce55490d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.300653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dbd06-334d-4aa9-b170-51dce55490d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.312492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815dbd06-334d-4aa9-b170-51dce55490d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-815dbd06-334d-4aa9-b170-51dce55490d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:15.317914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:15.318814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:15.319618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:15.322704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 466/928] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:15.325523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c490-72c3-414f-9386-da19925e661b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.365815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61082170-f7fc-4f8b-b0c1-466d623de8ef None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.366119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61082170-f7fc-4f8b-b0c1-466d623de8ef None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.366362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61082170-f7fc-4f8b-b0c1-466d623de8ef None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.367987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 467/929] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:15.377144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.377796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.378269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.402757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:15.404249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:15.429941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64c490-72c3-414f-9386-da19925e661b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.445003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:15.470541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.472594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.473253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.473794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.474869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:15.475373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:15.477959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:15.480394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 468/930] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:15.487822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 463/931] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:15.493537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.595086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ebe81-8678-40f6-a77b-ed7448070f5d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.617642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:15.633999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.634936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.635407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.635869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.646758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:15.648551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:15.664168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:15.665100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:15.665968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:15.665968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:15.665968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:15.668048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 469/932] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:15.674544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.750567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.782409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7d6ae-cd5b-4b04-aa8f-2e0ffd14d8fc None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.816522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.817202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.817478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.817748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.818388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:15.818687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:15.820111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:15.838125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 464/933] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:15.839464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.849034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:15.883905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cc884-f77e-4b44-b137-faf88d3c7b1c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.928083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.928964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.929828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.930332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.931325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:15.933328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:15.944249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:15.945226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 17 20:52:15.946092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:15.946942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01abf6-4547-4a36-b7f0-ed4e50c24222 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:15.949433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 470/934] 10.222.0.16 () {56 vars in 1132 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:15.955863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:15.990495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:15.991668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:15.992470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:15.993241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:15.996801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.001264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.003734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:16.016718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:16.018265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:16.019486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:16.022620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 465/935] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:16.028213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.074044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d84525-eac7-4db6-9711-2953b4cfb1db None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.100001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.117774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.118392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.118668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.118942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.119578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:16.119879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.121468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.128717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 471/936] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:16.134834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.150615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986ee98-da7a-4502-97c6-e72845a2b515 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.188439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.189083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.189366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.190172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.190922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:16.191228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.192866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.217995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 466/937] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 17 20:52:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:16.224237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.260862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.264900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4b331-0e77-46bd-90c7-7ef7400f6e8c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.312114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.312758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.313046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.313307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.314007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:16.314319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.317283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc51a3-504f-4c81-b0dd-18aaefd21bf4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.324717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 472/938] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:16.330892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d713-ca2a-4f0c-b92f-3a0c2ee072f3 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.331220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d713-ca2a-4f0c-b92f-3a0c2ee072f3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.331445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d713-ca2a-4f0c-b92f-3a0c2ee072f3 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.332876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 473/939] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:16.333443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d713-ca2a-4f0c-b92f-3a0c2ee072f3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.344043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f6a08-e866-4941-bb7c-0a97a8e285bc None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.370636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.382540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.383165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.383442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.383714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.394639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.396105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-39660bc0-348e-4c57-afa6-ee7c98885424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:16.411466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:16.412633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:16.413728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:16.413728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:16.413728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:16.417069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 467/940] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:16.418526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.418815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.419092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.436790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:16.438039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:16.465884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d713-ca2a-4f0c-b92f-3a0c2ee072f3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.506684 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec66f-f0ef-49b9-b7d0-5a11fa0e4af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.507350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec66f-f0ef-49b9-b7d0-5a11fa0e4af9 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.507620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec66f-f0ef-49b9-b7d0-5a11fa0e4af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.507890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec66f-f0ef-49b9-b7d0-5a11fa0e4af9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.508551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ec66f-f0ef-49b9-b7d0-5a11fa0e4af9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.510157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ec66f-f0ef-49b9-b7d0-5a11fa0e4af9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.514421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 468/941] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:16.515509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.546975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 474/942] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:16.594892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bc569-162e-454d-b35e-c7fc60b57c60 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.595567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bc569-162e-454d-b35e-c7fc60b57c60 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.596147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bc569-162e-454d-b35e-c7fc60b57c60 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.597984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 475/943] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:16.604224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.604564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.604892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.626050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:16.628257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:16.636408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1c6ca-b16f-40c9-bce1-47a9d7f7c2ee None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.645767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.648869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.694003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.694793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.695234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.695603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.696358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:16.696804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.699135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.714941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 469/944] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:16.720828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.727393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 476/945] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:16.817979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.828862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a523156e-4c46-4d8c-8b8a-9128f563303c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.871331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.872613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.873127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.873603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.882547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.883983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:16.897750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:16.898655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:16.899569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 470/946] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:16.906050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:16.933449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:16.942537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af63b12-9a23-4acf-ac38-0100b0c0aa00 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:16.980181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:16.981168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:16.981807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:16.982354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:16.983187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:16.984858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:16.988386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING keystone.server.flask.common [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 17 20:52:17.007547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.sql.core [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 20:52:17.007547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 20:52:17.007547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [parameters: {'id': 'b7f4517049c549748548da9e871e1c73', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 20:52:17.007547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81335) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 20:52:17.013339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8597dd8-6e94-4bb8-8f4c-83e5109acd45 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b7f4517049c549748548da9e871e1c73.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b7f4517049c549748548da9e871e1c73. Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:52:17.016429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result.read() Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 17 20:52:17.017407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application yield resource Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise self.value Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application session.commit() Sep 17 20:52:17.018880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.session.flush() Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application compat.raise_( Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 20:52:17.019987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 20:52:17.021122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application result.read() Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:52:17.022085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [parameters: {'id': 'b7f4517049c549748548da9e871e1c73', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.023066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:17.023964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b7f4517049c549748548da9e871e1c73. Sep 17 20:52:17.025500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.031733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 477/947] 10.222.0.16 () {60 vars in 1140 bytes} [Sat Sep 17 20:52:16 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:17.031733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.042012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:17.049938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.050536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.050912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.051194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.051889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:17.052187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.053653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.067175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 471/948] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:17.069834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.146514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12981595-352e-4635-b9d3-2f2ef960b694 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.164236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:17.184485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.185114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d2caa-06cb-4e74-9014-eda5d0c0cafd None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.185222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.185408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.185680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.186259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.188833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:17.198917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:17.199897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:17.201621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 478/949] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:17.202479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.226524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.227180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.227463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.227740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.235209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.237351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:17.248508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:17.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:17.250406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 472/950] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:17.250928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.282778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:17.319594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5dab5b-b72f-4006-b945-fe60b4b6f187 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.356201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.356878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.357152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.357418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/6a042e5b3cc04a83a52b61b1e0bbac33/roles/c722fc4c0c804be2a4eb9cadb419c454` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.359001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81951ac6-d015-48f1-aa17-ed054277a38d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.359666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=6a042e5b3cc04a83a52b61b1e0bbac33, role_id=c722fc4c0c804be2a4eb9cadb419c454)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.361136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.386907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 479/951] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:52:17 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/6a042e5b3cc04a83a52b61b1e0bbac33/roles/c722fc4c0c804be2a4eb9cadb419c454 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:17.386907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.425174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.426145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.427497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.427886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.428623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:17.429021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.430500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.438131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 473/952] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:17.445110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.458871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:17.519698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e4dcb-bace-4fa8-9cc6-3c1fdb966871 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.555895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:17.563408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.564608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.565940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.566492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.567347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:17.567866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.569721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.595443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 480/953] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:17.598927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8741c9-1e01-4d26-a3be-3294d6301552 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.640924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.641880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.642337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.643357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.644187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:17.644664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.646238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3227c-40d9-4787-b27b-5054dbed5bd3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.652595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 474/954] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:17.658668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:17.770970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bad12-aa46-4722-a9c2-f6ad05c7309a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:17.807593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b76f3c-6335-40e9-b3f1-452fb6b69aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:17.808451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b76f3c-6335-40e9-b3f1-452fb6b69aca None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:17.808747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b76f3c-6335-40e9-b3f1-452fb6b69aca None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:17.809016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b76f3c-6335-40e9-b3f1-452fb6b69aca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:17.809682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b76f3c-6335-40e9-b3f1-452fb6b69aca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:17.811042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b76f3c-6335-40e9-b3f1-452fb6b69aca None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:17.835528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 481/955] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:17 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:18.833763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac889ac2-c48b-4e3d-96aa-f2236647a2d2 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:18.834308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac889ac2-c48b-4e3d-96aa-f2236647a2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:18.834308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac889ac2-c48b-4e3d-96aa-f2236647a2d2 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:18.835853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 475/956] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:18.841338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:18.841722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:18.841907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:18.861108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:18.862869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:18.905314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:18.938299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 482/957] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:19.220048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac889ac2-c48b-4e3d-96aa-f2236647a2d2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:19.292463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:19.326882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac889ac2-c48b-4e3d-96aa-f2236647a2d2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:19.365302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:19.365929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.366205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.366475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.367226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:19.368586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:19.378344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:19.380323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:19.381502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 476/958] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:19.385751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:19.495520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e2cf-fc3f-44c9-8eef-e5b3b4d4d595 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:19.536408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:19.537046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.537321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.537594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.538223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:19.538518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:19.539889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e8e57-9a62-46f4-8585-42afc4ccef7d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:19.545151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 483/959] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:19.551184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:19.661516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9dd2a8-ce54-4bc9-8fe8-19c19d187267 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:19.700066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:19.700666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.700975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.701396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.702035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:19.702332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:19.703687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84cefa-6d51-47ee-a694-2a3c13437683 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:19.705258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:19.709276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 477/960] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:19.727994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de2420-0611-4ae9-a542-668eac48196d None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.728293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de2420-0611-4ae9-a542-668eac48196d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.728626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de2420-0611-4ae9-a542-668eac48196d None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.730280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 484/961] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:19.735992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcefad-68ba-4aec-af7c-0fa97c22dba5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.737629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcefad-68ba-4aec-af7c-0fa97c22dba5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.738136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcefad-68ba-4aec-af7c-0fa97c22dba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.758679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4edcefad-68ba-4aec-af7c-0fa97c22dba5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:19.760236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4edcefad-68ba-4aec-af7c-0fa97c22dba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:19.842097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 478/962] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:19.984097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e1984-e87d-49cd-b8f5-bded0b4cee6e None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.984635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e1984-e87d-49cd-b8f5-bded0b4cee6e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.984635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e1984-e87d-49cd-b8f5-bded0b4cee6e None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:19.986921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 485/963] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:19.992108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:19.992395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:19.992689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.010975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:20.012086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:20.075568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837a488-d986-406d-8493-73f9a918f110 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.075850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837a488-d986-406d-8493-73f9a918f110 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.076322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837a488-d986-406d-8493-73f9a918f110 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.078117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 486/964] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:20.083789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 479/965] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:20.087742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.087742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.087742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.104699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:20.106151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:20.112717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.144957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:20.183041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:20.203143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 487/966] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:20.230018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156699a6-f4f0-40ff-9234-84da9122e5d5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.269043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.270082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.270605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.271132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.282451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:20.288141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:20.289131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:20.290190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.292508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 480/967] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:20.298062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.372405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.406107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca690464-5543-463d-aba1-30cd4840ce0f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.443128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.443739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.444014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.444286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.445078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:20.445377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.446760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:20.456730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 488/968] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:20.462169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.493329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4539c6-31c8-4278-8eb0-a45551092361 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.497807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:20.516886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:20.537183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.537701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.537985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.538255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.538907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.540597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:20.543934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING keystone.server.flask.common [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 17 20:52:20.558269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.sql.core [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 20:52:20.558269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 20:52:20.558269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [parameters: {'user_id': 'd650e407f055401eb6a137242ead75d0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 20:52:20.558269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81336) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 17 20:52:20.575761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46161348-0752-46ea-b61a-2923d962dadd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application result.read() Sep 17 20:52:20.576487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:52:20.577412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application yield resource Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application next(self.gen) Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self.session.rollback() Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self.force_reraise() Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise self.value Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application session.commit() Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 17 20:52:20.578401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._prepare_impl() Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self.session.flush() Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._flush(objects) Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application compat.raise_( Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application flush_context.execute() Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rec.execute(self) Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 17 20:52:20.579550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 17 20:52:20.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application result = self._query(query) Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application conn.query(q) Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application result.read() Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 17 20:52:20.582336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd650e407f055401eb6a137242ead75d0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:20.583265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 17 20:52:20.584103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 481/969] 10.222.0.16 () {60 vars in 1131 bytes} [Sat Sep 17 20:52:20 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:20.587823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.621175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.621800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.622084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.622359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.634458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.635914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:20.650990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:20.652034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:20.653259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:20.653259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:20.653259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:20.653259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 489/970] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:20.657935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.698414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6030ae-e0d4-4569-9d14-04bb3d1b472a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.737318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.737944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.738219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.738489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.739066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.740625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:20.743654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:20.750298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:20.751269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:20.752619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:20.757157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 482/971] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:20.758187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.768388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db819cb7-e94b-4249-bb6e-d7452ea9a4b1 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.804036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:20.806620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.807265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.807553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.807793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.816391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.817868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:20.829609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:20.831504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 490/972] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:20.832818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.871226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e13d3b-0725-46ae-bd1a-3567cd917707 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.907142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.907744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.908026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.908368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.909036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:20.909335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.910693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:20.925259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 483/973] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:20.926004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:20.948081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fe673-845f-4ce4-9eba-638287d41b0f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:20.990676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:20.991344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:20.991708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:20.992054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:20.992788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:20.993159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:20.994595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.004920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 491/974] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:21.004920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.007204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.031207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bacdc-7ce6-4d9d-89f0-8b0ac98d87fb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.034906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.070879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.071392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.071667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.071938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.072625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:21.072947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.074360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.110537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 484/975] 10.222.0.16 () {56 vars in 1101 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:21.115540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca06d88-b5db-4e25-83fe-d70cba9d9285 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.121951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.157484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.158705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.159166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.159654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.168466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.170411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:21.184273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:21.185320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 17 20:52:21.186250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:21.194789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 492/976] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:52:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:21.196086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.246176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf48161-f8d9-431f-b82e-31b64ee1cc85 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.276172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.295161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.295779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.296058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.296331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.297002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:21.297312 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.299953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.311641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 485/977] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:21.314088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.332932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39609f68-11e0-4122-bd13-bcba03c701c0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.344010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.384989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.385603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.385884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.386160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.386896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:21.387204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.395368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.417934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 493/978] 10.222.0.16 () {56 vars in 1149 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:21.425129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.445538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2df8-ec38-4a21-aea5-67ac04b79a9f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.485887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.487308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.487612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.487852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.488807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] RBAC: Adding query filter params (interface=admin, service_id=b52980edcb644480a5a6c9cb021d2e9b, region_id=RegionOne) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:21.489185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.490505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.500008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 486/979] 10.222.0.16 () {56 vars in 1249 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/endpoints?service_id=b52980edcb644480a5a6c9cb021d2e9b&interface=admin®ion_id=RegionOne => generated 193 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:21.501073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.541549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf32d13-25e2-4fc2-8dea-6ebbedd8cc91 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.566789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.582248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cb60b-2697-488d-9a34-f5aad8dfb492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.583051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb60b-2697-488d-9a34-f5aad8dfb492 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.583738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb60b-2697-488d-9a34-f5aad8dfb492 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.584117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb60b-2697-488d-9a34-f5aad8dfb492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.597487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.608389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be095-7c16-41a3-864d-c358639c990c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.648263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.649072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.649446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.649813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.650513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.651964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.657004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cb60b-2697-488d-9a34-f5aad8dfb492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.659183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cb60b-2697-488d-9a34-f5aad8dfb492 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.659783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 487/980] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 20:52:21.778707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 494/981] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:21.779917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.893804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cbe7e-1888-4e9b-8a2f-c56eb38c3f73 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:21.928977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:21.929615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.929893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.930201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.941072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:21.942519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:21.955411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a74218-2cbd-4cf7-af18-f0dc6e35c5a8 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.955710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a74218-2cbd-4cf7-af18-f0dc6e35c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.955986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a74218-2cbd-4cf7-af18-f0dc6e35c5a8 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.957500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 495/982] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:21.958653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:21.959745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:21.960933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:21.960933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:21.960933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:21.960933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 488/983] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:21.962743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:21.963111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:21.963361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:21.965440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:21.978562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:21.983453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:21.984555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:22.004044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:22.059139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 496/984] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:22.080510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b54404-5f90-4c90-b94e-bfd2cea8afa1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:22.115256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:22.115858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:22.116137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:22.116409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:22.117075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:22.117377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:22.118762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:22.131229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 489/985] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:22.136799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:22.239715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec517a-708c-4db5-8e17-59c7acd7e6df None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:22.274042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:22.274669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:22.274944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:22.275220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:22.281824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:22.283308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:22.294319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:22.295269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:22.296250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:22.297303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 497/986] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:22.302039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:22.340270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:22.355964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:22.399207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:22.437785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c15dd-7edc-48d1-ae33-35d25ecf4352 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:22.450595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0e0c2-d85f-4078-b227-2634dda14f69 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:22.475494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:22.476390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:22.476879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:22.477156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:22.477844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:22.478148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:22.479562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:22.486954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b1e73-c6c1-403a-8197-ea7476dd6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:22.488015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b1e73-c6c1-403a-8197-ea7476dd6350 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:22.488287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b1e73-c6c1-403a-8197-ea7476dd6350 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:22.488554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b1e73-c6c1-403a-8197-ea7476dd6350 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:22.489273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b1e73-c6c1-403a-8197-ea7476dd6350 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:22.490678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b1e73-c6c1-403a-8197-ea7476dd6350 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:22.492174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 490/987] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:22.498212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:22.512725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 498/988] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:22 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:22.609623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065b857-ceed-45c7-bdd1-4f5d50334bc7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:22.638838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:22.656241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:22.656890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:22.657167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:22.657435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:22.658355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] RBAC: Adding query filter params (role.id=c722fc4c0c804be2a4eb9cadb419c454, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=6a042e5b3cc04a83a52b61b1e0bbac33) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:22.658792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:22.660248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277be6e1-e661-4aa8-9e74-0705f875e4f4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:22.667035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 491/989] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:22 2022] GET /identity/v3/role_assignments?user.id=6a042e5b3cc04a83a52b61b1e0bbac33&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=c722fc4c0c804be2a4eb9cadb419c454 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:23.866968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4b49f-77d8-491a-b60a-f6b4cc45be18 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:23.867510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4b49f-77d8-491a-b60a-f6b4cc45be18 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:23.867510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4b49f-77d8-491a-b60a-f6b4cc45be18 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:23.869977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 499/990] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:23.874553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:23.874957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:23.875153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:23.894721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:23.896221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:23.961335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 492/991] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:24.233434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4b49f-77d8-491a-b60a-f6b4cc45be18 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:24.304876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:24.343342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4b49f-77d8-491a-b60a-f6b4cc45be18 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:24.381387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:24.382018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.382294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.382570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.383158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:24.384527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:24.396975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:24.398068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:24.401695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 500/992] 10.222.0.16 () {56 vars in 1126 bytes} [Sat Sep 17 20:52:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:24.404470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:24.507176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da69b9-773d-40a6-990b-4792afcade09 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:24.523898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:24.543010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:24.543616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.543897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.544172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.544827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:24.545131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:24.546496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de00eb-6f74-4723-b99b-02096b14080b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:24.556542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 493/993] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:24.558058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:24.665382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c955-c851-4237-8db8-3144d4983ac0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:24.699829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:24.700434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.700740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.701065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.704720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:24.704720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:24.705113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c6234-3ee3-4e0e-9455-fe558d78df63 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.705113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c6234-3ee3-4e0e-9455-fe558d78df63 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.705420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:24.705578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c6234-3ee3-4e0e-9455-fe558d78df63 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.705578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 494/994] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:24.706655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:24.710939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 501/995] 10.222.0.16 () {56 vars in 1122 bytes} [Sat Sep 17 20:52:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:24.711852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c11983-5afb-4283-8654-0b5854f14d35 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.712536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c11983-5afb-4283-8654-0b5854f14d35 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.713009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c11983-5afb-4283-8654-0b5854f14d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.716182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:24.737362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-91c11983-5afb-4283-8654-0b5854f14d35 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:24.738777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-91c11983-5afb-4283-8654-0b5854f14d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:24.825040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3f7d7-e48b-4a81-b652-c69e6c12fcf6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:24.833006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 495/996] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:24.861516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:24.862240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.862518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.862793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.863460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:24.864856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:24.869695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f2c63-fee6-4183-afa5-92f37ca2479e None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.869995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f2c63-fee6-4183-afa5-92f37ca2479e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.870269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f2c63-fee6-4183-afa5-92f37ca2479e None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.871735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 496/997] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:24.876481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:24.877069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:24.877069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:24.892968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 502/998] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:24.906059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:24.907380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:24.988711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 497/999] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:25.100365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.161139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.205438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce6e50-a8ac-46f0-811b-616a9450e2a5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.238600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.239310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.239649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.239983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.240627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.242129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:25.250860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:25.252513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:25.253586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:25.256802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 503/1000] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:25.258074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.260999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.294807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.365957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28260e2b-aa55-4109-8f54-7a34a7b800bc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.370699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b39ea-d93d-4191-a17c-b5a0ad043ed3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.403356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.404391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.405056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.405411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.405711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] PATH_INFO: `/v3/services/image` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.406296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.407652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.416448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:25.416957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:25.418096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:25.419277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 498/1001] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:25.420620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.421070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.421477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.422242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:25.422781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.423829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.424510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.435472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 504/1002] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:25.436533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.527430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.531252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e384bbd-1500-4e23-90bb-354dff191829 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.540870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d0258-1ebc-4a5d-b281-e22b3377e60b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.570646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.571510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.571867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.572152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.572991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:25.573292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.574757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.574857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.575446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.575725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.575995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.576614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:25.576944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.578291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.581239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 499/1003] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:25.588354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 505/1004] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:25.588497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.589417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.645462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.694773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628a988-eacb-48b1-82f0-13e1c8b8a938 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.697595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67797d6-5d2b-422b-bafa-c69297c28ac8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.730116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.730958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.731336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.731633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.732335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] RBAC: Adding query filter params (interface=public, service_id=59601360b17c4f6e9f67fe97060d175f, region_id=RegionOne) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:25.732642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.733497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.734030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445d3de-d65a-463e-9dba-73c1f31f27a0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.734170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.734543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.734894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.735583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] RBAC: Adding query filter params (type=image) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:25.735941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.738108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.752974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 500/1005] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:25.753225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 506/1006] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/endpoints?service_id=59601360b17c4f6e9f67fe97060d175f&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:25.753359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:25.782407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:25.865273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bad71-13fc-49bc-8e70-a8fea490051f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:25.915089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa8194-6ed4-40d9-813a-4dd42368acb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:25.915721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa8194-6ed4-40d9-813a-4dd42368acb7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:25.915969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa8194-6ed4-40d9-813a-4dd42368acb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:25.916175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa8194-6ed4-40d9-813a-4dd42368acb7 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:25.916777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa8194-6ed4-40d9-813a-4dd42368acb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:25.919322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa8194-6ed4-40d9-813a-4dd42368acb7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:25.932964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 501/1007] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:52:25 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:27.130683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b489e5b-5008-451a-bff9-9f87ea68f47c None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.131225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b489e5b-5008-451a-bff9-9f87ea68f47c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.131225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b489e5b-5008-451a-bff9-9f87ea68f47c None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.132778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 507/1008] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:27.137812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.138193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.138374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.157563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:27.159694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:27.228187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 502/1009] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:27.504343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b489e5b-5008-451a-bff9-9f87ea68f47c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:27.608566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b489e5b-5008-451a-bff9-9f87ea68f47c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:27.625534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:27.644720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782b64f-5e08-48bf-8475-8f170f308a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:27.645333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b64f-5e08-48bf-8475-8f170f308a24 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.645617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b64f-5e08-48bf-8475-8f170f308a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.645891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b64f-5e08-48bf-8475-8f170f308a24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.647010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782b64f-5e08-48bf-8475-8f170f308a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:27.648448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782b64f-5e08-48bf-8475-8f170f308a24 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:27.652702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 508/1010] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:27.655775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:27.681562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:27.759974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363f727-d163-4b08-bd5c-fae470018920 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:27.794718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264500e-c2f6-4573-83e6-532e449994e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:27.795318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264500e-c2f6-4573-83e6-532e449994e0 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.795593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264500e-c2f6-4573-83e6-532e449994e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.795863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264500e-c2f6-4573-83e6-532e449994e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.796520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264500e-c2f6-4573-83e6-532e449994e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:27.798011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264500e-c2f6-4573-83e6-532e449994e0 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:27.836700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 503/1011] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:27 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:27.931903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318981e4-60b5-4e3f-afe2-a0947278e1e2 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.932207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318981e4-60b5-4e3f-afe2-a0947278e1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.932480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318981e4-60b5-4e3f-afe2-a0947278e1e2 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.934031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 509/1012] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:27.939662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ee47-de65-4447-9fe1-893e7f0ad442 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:27.939980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ee47-de65-4447-9fe1-893e7f0ad442 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:27.940218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677ee47-de65-4447-9fe1-893e7f0ad442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:27.959963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e677ee47-de65-4447-9fe1-893e7f0ad442 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:27.961213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e677ee47-de65-4447-9fe1-893e7f0ad442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:28.041133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 504/1013] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:27 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:28.121090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cf57c-c0eb-4fd4-b768-434b14d066ca None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.121411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cf57c-c0eb-4fd4-b768-434b14d066ca None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.121641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cf57c-c0eb-4fd4-b768-434b14d066ca None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.123008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 510/1014] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:28.129053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.129352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.129631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.136859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:28.148764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:28.150524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:28.222153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 505/1015] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:28 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:28.306002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cf57c-c0eb-4fd4-b768-434b14d066ca None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:28.423909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cf57c-c0eb-4fd4-b768-434b14d066ca None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:28.461232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:28.462182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.462682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.463153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.464005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:28.465617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.482090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 511/1016] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:28 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:28.487773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:28.597575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96339c-5c02-4aa7-b60c-3a5db9058403 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:28.635082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:28.642790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:28.643449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.643722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.643988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.644568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:28.645969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:28.655061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:28.656093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:28.658149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 506/1017] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:28.662100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:28.709163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:28.765922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089f228-9119-46da-b95a-d88b30779b00 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:28.801008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:28.801606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.801880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.802149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.802768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:28.803063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:28.804418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.813932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 512/1018] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:28.815652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:28.914871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f388f6-d3aa-4714-8bef-87968850735c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:28.948280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:28.948918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:28.949202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:28.949476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:28.950103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:28.950406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:28.951764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d266ef-882c-4327-a529-ce0c9198bda2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:28.957168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 507/1019] 10.222.0.16 () {56 vars in 1112 bytes} [Sat Sep 17 20:52:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:28.962642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:29.064303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bdc04-d5be-4554-a562-b17ea41ff9fc None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:29.073251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:29.100861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936fce5-228e-492d-8ea5-d75d278da4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:29.101729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936fce5-228e-492d-8ea5-d75d278da4f4 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:29.102187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936fce5-228e-492d-8ea5-d75d278da4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:29.102462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936fce5-228e-492d-8ea5-d75d278da4f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:29.103142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936fce5-228e-492d-8ea5-d75d278da4f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:29.104501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936fce5-228e-492d-8ea5-d75d278da4f4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:29.124881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 513/1020] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:30.064046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d184dc-d1cd-46f2-9612-34ae0722912d None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.064566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d184dc-d1cd-46f2-9612-34ae0722912d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.064644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d184dc-d1cd-46f2-9612-34ae0722912d None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.066050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 508/1021] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:30.071330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.071747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.071908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.091317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:30.092560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:30.168958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 514/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:30.435682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d184dc-d1cd-46f2-9612-34ae0722912d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.499121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:30.544170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d184dc-d1cd-46f2-9612-34ae0722912d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.581432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3291b79-9d60-41f1-9419-4b4cfe2c0a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.582077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3291b79-9d60-41f1-9419-4b4cfe2c0a19 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.582355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3291b79-9d60-41f1-9419-4b4cfe2c0a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.582625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3291b79-9d60-41f1-9419-4b4cfe2c0a19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.583824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3291b79-9d60-41f1-9419-4b4cfe2c0a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.585323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3291b79-9d60-41f1-9419-4b4cfe2c0a19 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.587203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 509/1023] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:30.592735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:30.678394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800ba6-35b8-4ae7-bff1-35217515e682 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.678573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800ba6-35b8-4ae7-bff1-35217515e682 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.678852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d800ba6-35b8-4ae7-bff1-35217515e682 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.680185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 510/1024] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:30.685126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.685475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.685822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.686407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:30.699616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcea94-9952-454a-a3c1-041edd2c0117 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:30.705422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:30.706874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:30.735465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:30.736738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:30.737013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:30.737286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:30.737947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:30.739376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:30.761408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 515/1025] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:30.785067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 511/1026] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:31.051821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.156846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16132b-c9a2-4de4-8b59-3264604bc292 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.193463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.194093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.194373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.194644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.195222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:31.196662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:31.206125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:31.207148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:31.207965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 516/1027] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:31.214860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.273021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:31.321393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b9e62-245f-4afe-8ae8-c22c22e0f894 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.354911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.355504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.355778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.356044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.356663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] RBAC: Adding query filter params (name=image) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:31.356981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:31.358332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f73a96-12a8-42fb-aa7f-0b33cf00e521 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:31.368447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 512/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:31.370009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.390250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:31.393005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa091a59-77b5-44d3-925b-5c0ecf54f69f None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.393309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa091a59-77b5-44d3-925b-5c0ecf54f69f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.393586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa091a59-77b5-44d3-925b-5c0ecf54f69f None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.394935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 513/1029] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:31.399972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.400253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.400529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.418743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:31.419918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:31.487530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39b9e7-eaef-492a-b14e-930fc55b27f5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.497918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 514/1030] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:31.525563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.526403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.526681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.526953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.527581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:31.527881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:31.529280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:31.535046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 517/1031] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:31.540888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.592013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:31.643587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1fd11-bff3-4750-81c8-e34f5f946fc8 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.677703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.678300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.678576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.678845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.679533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] RBAC: Adding query filter params (interface=public, service_id=e3882d4a788f4115aa85b2f27696657d, region_id=RegionOne) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:31.679834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:31.681227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:31.686352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 515/1032] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/endpoints?service_id=e3882d4a788f4115aa85b2f27696657d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:31.691965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.769190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:31.770238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:31.797498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9120e66-272d-4822-b977-60fa32e1efc4 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.832506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.833128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.833406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.833674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.834325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:31.835698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:31.842922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 518/1033] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:52:31.871644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758e124-10ae-4e9e-9d3c-24648327456d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:31.889287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:31.913446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:31.914088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:31.914360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:31.914638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:31.926534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:31.931421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:31.932386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:31.933583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:31.934664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 516/1034] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:31.939436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.047260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99f967-1796-42e4-b4c8-41af45b54010 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.082052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.082657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.082930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.083198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.083859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:32.084154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.085538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:32.090691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:32.095285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 519/1035] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:52:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:32.101134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.207305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6fc05-b246-4873-b04e-af893e63eb4a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.241878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.243231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.243708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.244158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.254970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.256601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:32.271198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:32.272224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:32.273150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:32.273150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:32.273150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:32.276751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 517/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:32.278712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.385019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8deb-25da-46af-b7ba-649df9dd8ed0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.419738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.420331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.420606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.420902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.421539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:32.421832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.423184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:32.435402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 520/1037] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:32.441395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.469232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:32.542982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d59a8-32e3-45bb-ba1f-2bd36fa5e36c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.576982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.577606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.577880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.578147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.586153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.587574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:32.599025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:32.599984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:32.600956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:32.604969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 518/1038] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:52:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:32.606278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.670141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:32.712658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4dda2e-5860-4c5b-9fca-db456b7f918f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.749755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.750356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.750632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.750903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.751729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:32.752034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.753564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a60afd-58a4-410a-96ae-db395794f83b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:32.775302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 521/1039] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:52:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:32.776494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:32.865075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:32.881448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e03c4-956d-44df-bf98-0fa6305290b8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:32.920431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:32.920989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:32.921372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:32.921679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:32.990752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:32.992205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.007599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23dce4-6dd0-49fc-a1e9-dcc2eb37e345 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.008314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23dce4-6dd0-49fc-a1e9-dcc2eb37e345 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.008597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23dce4-6dd0-49fc-a1e9-dcc2eb37e345 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.010111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 522/1040] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:33.014866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.015174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.015447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.035174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:33.036435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:33.083510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.108494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 519/1041] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:32 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 20:52:33.109550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.115787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 523/1042] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:33 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:33.211193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9acaa1-8ec7-42f2-8fe4-67b6b59755c7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.245841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.246436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.246713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.246978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.256833 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.258243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:33.272658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:33.273583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:33.274888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:33.274888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:33.274888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:33.274888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 520/1043] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:33.281417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.329319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.385339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874136-4741-4d1a-98d5-53ce10e2f16d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.390935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.422452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.423430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.423974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.424449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.425117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:33.425420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.426793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.436686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.443276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 524/1044] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:33.444571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.507242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a07c00-9d50-4e9c-a444-0abab61d5be9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.543291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.543910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.544191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.544462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.552772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5916c47-75cf-4f91-b98f-b5ccc606adb0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.555233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:33.560037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:33.561024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:33.561913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 521/1045] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:33.564365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.567896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.593510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.594234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.594576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.594862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.601712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.603167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:33.614436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:33.615370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:33.617203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 525/1046] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:33.621241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.669938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.674225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcfeb75-57c2-44a7-902e-9cf63cb60e39 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.712241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.712882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.713161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.713429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.714476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:33.714795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.716149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.725930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 522/1047] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:33.726078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d424eb49-47dd-4e86-9f04-ec48715d1eb0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.731522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.765777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.767611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.768108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.768604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.769527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:33.770033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.771679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.784707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 526/1048] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:33.790479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.820477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.840415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593814b1-71ca-4f14-89d0-b6abf74256ca None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.878452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.879100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.879379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.879652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] PATH_INFO: `/v3/users/3573339b8014493383118dd9438d487d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.893458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] RBAC: Authorizing `identity:get_user(user_id=3573339b8014493383118dd9438d487d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.894926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.896606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d05da-ed1e-4bb0-a444-82678d6075e7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:33.896894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 523/1049] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/users/3573339b8014493383118dd9438d487d => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:33.902849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:33.926658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:33.941227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:33.941825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:33.942108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:33.942384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:33.943123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] RBAC: Adding query filter params (role.id=c722fc4c0c804be2a4eb9cadb419c454, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec86be5855254c8abc10fb636add2749) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:33.943434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:33.944900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1707b-bdab-4a35-b36f-92b519859e48 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:33.950458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 527/1050] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/role_assignments?user.id=ec86be5855254c8abc10fb636add2749&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=c722fc4c0c804be2a4eb9cadb419c454 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:34.014955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13054b-5b7e-415c-9ba3-c9b1a9b257f6 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.028639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f17f66-c2bf-43dc-aa0a-46dd631c8699 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.028858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f17f66-c2bf-43dc-aa0a-46dd631c8699 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.029248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f17f66-c2bf-43dc-aa0a-46dd631c8699 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.030716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 528/1051] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:34.036743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.037052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.037297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.056726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.057347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.057624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.057726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:34.057893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.059020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:34.137444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 529/1052] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:34.138239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.139697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.157460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:34.240188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 524/1053] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 17 20:52:34.246234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:34.270749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:34.352762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d1ce2-acd0-4560-8ec2-9c5cab868576 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.399021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.399645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.399919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.400199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.401417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] RBAC: Authorizing `identity:get_project(project_id=b34ded98138f45549fbde2161bfa2164)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.402855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.409456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 530/1054] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:52:34 2022] GET /identity/v3/projects/b34ded98138f45549fbde2161bfa2164 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:34.411813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:34.416439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:34.535392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:34.547734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb168c-b721-4377-9ede-9fd854e88638 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.549609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed79739-4b18-4d8c-b349-fdb6b1fd4d58 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.569151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:34.589320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1014f-2567-422c-9436-860474958332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.590066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1014f-2567-422c-9436-860474958332 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.590422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1014f-2567-422c-9436-860474958332 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.590527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.590709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1014f-2567-422c-9436-860474958332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.591082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.591360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.591463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1014f-2567-422c-9436-860474958332 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=b34ded98138f45549fbde2161bfa2164, user.id=3573339b8014493383118dd9438d487d) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:34.591633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.591740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1014f-2567-422c-9436-860474958332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.592244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.593147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1014f-2567-422c-9436-860474958332 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.593635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.608789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 525/1055] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:34 2022] GET /identity/v3/role_assignments?user.id=3573339b8014493383118dd9438d487d&scope.project.id=b34ded98138f45549fbde2161bfa2164&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:34.608992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:34.610185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:34.611741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 531/1056] 10.222.0.16 () {56 vars in 1105 bytes} [Sat Sep 17 20:52:34 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:34.612594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1014f-2567-422c-9436-860474958332 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:34.759826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1014f-2567-422c-9436-860474958332 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.798074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.799002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.799357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.799748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.800414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] RBAC: Adding query filter params (name=image) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:34.800747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.802351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.809649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 526/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:34.816277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:34.923302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568687d-7761-45f1-a25b-51a5e6adb7b6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:34.963152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:34.963759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:34.964032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:34.964301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:34.964954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] RBAC: Adding query filter params (type=image) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:34.965253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:34.967342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb686617-1d09-4106-9acb-64737abb7bff None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:34.977844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 532/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:34.978793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:35.036199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:35.092324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9184c2-f15e-4336-a66e-5820e0a2c1ef None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:35.127937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1b313-88ef-4518-a365-d04a8c492b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:35.128550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1b313-88ef-4518-a365-d04a8c492b13 None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:35.128858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1b313-88ef-4518-a365-d04a8c492b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:35.129127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1b313-88ef-4518-a365-d04a8c492b13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:35.129806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1b313-88ef-4518-a365-d04a8c492b13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:35.131307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1b313-88ef-4518-a365-d04a8c492b13 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:35.152750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 527/1059] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:52:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:52:36.321986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a15b1-133d-4d61-a1c0-53acdfbd58b9 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:36.323873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a15b1-133d-4d61-a1c0-53acdfbd58b9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:36.323873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a15b1-133d-4d61-a1c0-53acdfbd58b9 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:36.324139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 533/1060] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:36.330242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:36.330677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:36.330878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:36.349296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:36.350465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:36.427715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 528/1061] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:36.696215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a15b1-133d-4d61-a1c0-53acdfbd58b9 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:36.716900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:36.809695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a15b1-133d-4d61-a1c0-53acdfbd58b9 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:36.844614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766ea86-e5ae-46fc-8ee6-e72efd4555d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:36.845432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ea86-e5ae-46fc-8ee6-e72efd4555d4 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:36.845879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ea86-e5ae-46fc-8ee6-e72efd4555d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:36.846157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ea86-e5ae-46fc-8ee6-e72efd4555d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:36.856939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ea86-e5ae-46fc-8ee6-e72efd4555d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-3766ea86-e5ae-46fc-8ee6-e72efd4555d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:36.862477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:36.863479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:36.864375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:36.868642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 534/1062] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 17 20:52:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:36.870089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:36.876064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c350a1-84ee-4666-9a91-00a7d0b12f6e None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:36.877043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c350a1-84ee-4666-9a91-00a7d0b12f6e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:36.877435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c350a1-84ee-4666-9a91-00a7d0b12f6e None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:36.878799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 535/1063] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:36.883793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:36.884079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:36.884355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:36.899562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:36.902392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:36.903658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:36.976843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 536/1064] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:52:36.983089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fb9f3-a118-4943-8730-4f59ec06e6da None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.018332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.018929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.019202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.019569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.020226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:37.020520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.027958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.045958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 529/1065] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:52:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:37.047653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.058755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.152667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92047a-91d5-4252-b6c5-8bee6a217a76 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.187537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.188159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.188432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.189296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.199963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.201458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:37.216992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:37.217963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:37.223760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:37.223760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:37.223760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:37.223760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 537/1066] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:37.225514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.259817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.324526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.336796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55611b-4044-4e11-9502-c297e5d947f5 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.341199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.371876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.372621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.373058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.373495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.374600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:37.375118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.378404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e60166-bb6d-49ad-b5fb-6da52a9e21a3 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.391733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 530/1067] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:37.393810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb9b53-1eb3-4862-8cea-adec1c1bd727 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.394093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082ec4e-1966-4426-97b9-31a9ec90a2ab None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.394266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb9b53-1eb3-4862-8cea-adec1c1bd727 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.394515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb9b53-1eb3-4862-8cea-adec1c1bd727 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.396848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 531/1068] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:37.398531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb9b53-1eb3-4862-8cea-adec1c1bd727 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.428602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe340edb-3376-4ed4-b0d6-40038fd29a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.429253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe340edb-3376-4ed4-b0d6-40038fd29a5d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.429531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe340edb-3376-4ed4-b0d6-40038fd29a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.429803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe340edb-3376-4ed4-b0d6-40038fd29a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.440562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe340edb-3376-4ed4-b0d6-40038fd29a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-fe340edb-3376-4ed4-b0d6-40038fd29a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:37.445292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:37.446401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:37.447643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 538/1069] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:37.448488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.448488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.448488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.466528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:37.467716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:37.502657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb9b53-1eb3-4862-8cea-adec1c1bd727 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.534131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 539/1070] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:37.534957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.541756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.543164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.543442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.543711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.551682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.553265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:37.565942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:37.566920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:37.567864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 532/1071] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:37.572581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.584950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.596204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.647662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f787aec-1ae4-401b-a18a-8e7222d6bba5 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.675034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3b6d5-255f-41db-b477-475d64495dc5 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.681327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.681932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.682204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.682470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.683127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:37.683423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.684812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.693924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 540/1072] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:37.699350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.726578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.727233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.727454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.727719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.728442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:37.728770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.730303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.755953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 533/1073] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:37.759348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.818279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e27db5-b91e-4943-a560-46be5fbda90f None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.826913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.830845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:37.851789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.852410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.852703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.852990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] PATH_INFO: `/v3/users/3573339b8014493383118dd9438d487d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.854116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] RBAC: Authorizing `identity:get_user(user_id=3573339b8014493383118dd9438d487d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.855543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.862755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 541/1074] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/users/3573339b8014493383118dd9438d487d => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:37.862755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.862979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ac231-5995-4a0e-a2f8-b45c184e393c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.895616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.896237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.896515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.896825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:37.906735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:37.909084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:37.924061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:37.925160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:37.926060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:37.926060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:37.926060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:37.926060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 534/1075] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:37.926295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:37.961876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d96059-d674-4620-85b6-aa04590e8846 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:37.994968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:37.995567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:37.995845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:37.996118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.023961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914db7b-fa7d-45a2-9665-ce937f454f16 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.044490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:38.046809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:38.059576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.060204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.060481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.060787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.061898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] RBAC: Authorizing `identity:get_project(project_id=b34ded98138f45549fbde2161bfa2164)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.063737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.065430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 535/1076] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/projects/b34ded98138f45549fbde2161bfa2164 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:38.066182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.072705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.074092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.170707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef959b-dd72-4bea-9210-5374912aa139 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.171300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 542/1077] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:38.172116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.204940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.205529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.205803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.206072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.206704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:38.207000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.208348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.220741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 536/1078] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:38.221552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.255586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:38.270317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:38.275242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece7faf-8f2d-4ec4-a6ca-adbaf500605e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.308526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.309206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.309485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.309756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164/users/3573339b8014493383118dd9438d487d/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.311628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] RBAC: Authorizing `identity:create_grant(project_id=b34ded98138f45549fbde2161bfa2164, user_id=3573339b8014493383118dd9438d487d, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.313086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.320467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d580d-b08e-4874-b0f4-42c8a72047eb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.325627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 543/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:52:38 2022] PUT /identity/v3/projects/b34ded98138f45549fbde2161bfa2164/users/3573339b8014493383118dd9438d487d/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:38.326362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.361991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.362631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.362924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.363205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.374930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.376468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:38.394065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:38.395201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:38.403825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:38.403825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:38.403825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:38.403825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 537/1080] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:38.403825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.485085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde2491-8538-4991-b420-e979ea8ae737 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.519156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:38.522698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981999a-c9fb-4e78-bc6e-e80db7094831 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.526551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.527365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.527799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.528240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.537770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.539240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.541891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:38.552484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:38.553777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:38.554869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:38.557520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 544/1081] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:38.563171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.570302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.570918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.571196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.571473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.572122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:38.572423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.574369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.600792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 538/1082] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:38.681034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b097e86-14df-402b-a4a0-7e436dcdc930 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.727322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.727949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.728229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.728495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.729255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:38.729564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.731049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc34da5-8e30-4f41-b7e3-96e219550230 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.745530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 545/1083] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:38.752110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:38.859770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6c723-3170-47aa-bfcd-a3aec36b3e21 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:38.894988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83e0bb-1358-4693-8b4f-4ee179866de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:38.895647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83e0bb-1358-4693-8b4f-4ee179866de5 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:38.895924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83e0bb-1358-4693-8b4f-4ee179866de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:38.896196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83e0bb-1358-4693-8b4f-4ee179866de5 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec86be5855254c8abc10fb636add2749/roles/c722fc4c0c804be2a4eb9cadb419c454` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:38.898328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83e0bb-1358-4693-8b4f-4ee179866de5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=ec86be5855254c8abc10fb636add2749, role_id=c722fc4c0c804be2a4eb9cadb419c454)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:38.899756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83e0bb-1358-4693-8b4f-4ee179866de5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:38.914080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 539/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:52:38 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/ec86be5855254c8abc10fb636add2749/roles/c722fc4c0c804be2a4eb9cadb419c454 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 20:52:40.534996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be16463-f3c3-4285-ab9b-1c3f3460c731 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:40.535495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be16463-f3c3-4285-ab9b-1c3f3460c731 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:40.535600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be16463-f3c3-4285-ab9b-1c3f3460c731 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:40.537350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 546/1085] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:40.542887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343119-bea7-4099-b8a8-d48963046a46 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:40.543306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343119-bea7-4099-b8a8-d48963046a46 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:40.543478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343119-bea7-4099-b8a8-d48963046a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:40.566368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-61343119-bea7-4099-b8a8-d48963046a46 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:40.567563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-61343119-bea7-4099-b8a8-d48963046a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:40.582832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:40.641167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 540/1086] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:40.912575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be16463-f3c3-4285-ab9b-1c3f3460c731 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:40.976263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.017850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be16463-f3c3-4285-ab9b-1c3f3460c731 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.050807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c656e2b-0d5e-4721-b633-181d8952942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.051419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c656e2b-0d5e-4721-b633-181d8952942f None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.051691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c656e2b-0d5e-4721-b633-181d8952942f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.051956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c656e2b-0d5e-4721-b633-181d8952942f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.061752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c656e2b-0d5e-4721-b633-181d8952942f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:41.066716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbfc2a-f116-4447-bec0-80989ad69b77 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.066998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbfc2a-f116-4447-bec0-80989ad69b77 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6c656e2b-0d5e-4721-b633-181d8952942f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:41.067091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:41.068160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:52:41.069032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:41.069668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dbfc2a-f116-4447-bec0-80989ad69b77 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.069748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 547/1087] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:41.069817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 541/1088] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:52:41.073868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.073980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbfc2a-f116-4447-bec0-80989ad69b77 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.074150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.074423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.092332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:41.093608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:41.162977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 548/1089] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:41.176219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dbfc2a-f116-4447-bec0-80989ad69b77 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.209707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.210362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.210671 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.210942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.211601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:41.211903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.213296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.221204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 542/1090] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:41.227033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.275392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.323540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811d754-d66e-47ea-86b9-182cb9aada59 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.355816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.356437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.356740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.357018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] PATH_INFO: `/v3/users/3573339b8014493383118dd9438d487d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.358140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] RBAC: Authorizing `identity:get_user(user_id=3573339b8014493383118dd9438d487d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.359566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.361783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 549/1091] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/users/3573339b8014493383118dd9438d487d => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:41.367578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.383575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.440062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.467008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41296854-9066-475f-918b-c0ae426e5934 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.500635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.501877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.502154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.502426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.538978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4af30-4e54-467b-90b7-e4c1b799cc61 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.556239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.568683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.570158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.574223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.574841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.575113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.575380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.584185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.586207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:52:41.590871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:41.591852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:41.592716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 550/1092] 10.222.0.16 () {56 vars in 1120 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:41.598459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.681228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 543/1093] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:41.687103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.705098 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5b4ad-ec35-4e87-b6ab-bb7e525da40a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.738514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.739108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.739387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.739656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.740307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:41.740606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.742003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.749810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 551/1094] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:41.755656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.786388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9be4fd-c691-4af3-8a11-b7b8784d393a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.795577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.800189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.820960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.821577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.821857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.822125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.823222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] RBAC: Authorizing `identity:get_project(project_id=b34ded98138f45549fbde2161bfa2164)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.824629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.826278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 544/1095] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/projects/b34ded98138f45549fbde2161bfa2164 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:41.832491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.856086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ad096-a620-4485-aa2e-d0ec1e18e308 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.888766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.889391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.889673 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.889948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.901091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.902545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:41.916185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:41.917289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:41.918179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:41.918179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:41.918179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:41.918179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 552/1096] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:41.922983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:41.931481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2efc95-82ae-47ad-a496-ee74cf6059ab None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:41.965936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:41.966469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:41.966746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:41.967023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:41.967743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=b34ded98138f45549fbde2161bfa2164, user.id=3573339b8014493383118dd9438d487d) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:41.968051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:41.969534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:41.975130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 545/1097] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/role_assignments?user.id=3573339b8014493383118dd9438d487d&scope.project.id=b34ded98138f45549fbde2161bfa2164&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:42.016724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:42.026388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e336d1-eb2f-4188-b75f-e9d6dbf2d676 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:42.067364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:42.067993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:42.068274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:42.068549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:42.069253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:42.069558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:42.070985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:42.084616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 553/1098] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:42.090489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:42.139616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:42.195184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80103e5c-a338-4ae1-8e28-45d67dbbb348 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:42.227464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:42.228009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:42.228282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:42.228547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:42.236320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:42.237775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:52:42.248462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:42.249622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 17 20:52:42.250752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:52:42.252818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 546/1099] 10.222.0.16 () {56 vars in 1093 bytes} [Sat Sep 17 20:52:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:52:42.256045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:42.354360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b656174-7b51-43a6-bf7a-9e67ca3aa6ba None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:42.390982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:42.391576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:42.392059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:42.392326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:42.393079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:42.393379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:42.394794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:42.412799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 554/1100] 10.222.0.16 () {56 vars in 1133 bytes} [Sat Sep 17 20:52:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:42.418296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:42.517044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18700e81-209c-4af0-8b93-1145477c1ebb None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:42.519287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:42.550683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:42.551285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:42.551564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:42.551835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:42.617529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:42.618993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:42.712631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 547/1101] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:42.718421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:42.731123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:42.818577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04af91-fa2c-4c53-b3dd-f060d6c81350 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:42.852242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:42.852887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:42.853165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:42.853438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:42.863572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:42.865099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:52:42.882479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:42.883616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:52:42.884537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:52:42.884537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:52:42.884537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:42.884537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 555/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:42.889298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:42.893084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:42.985527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee8014-eeb3-4e33-babc-94f9095ceff4 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:43.017750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:43.018327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:43.018602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:43.018870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:43.019499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:43.019796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:43.021188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:43.031891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 548/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:43.038324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:43.098903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:43.137601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6336e27-b501-46cd-9a19-5f9153efaab9 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:43.170352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:43.170962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:43.171238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:43.171507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:43.177984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:43.179426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:52:43.191337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:52:43.192215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:52:43.193441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 556/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:52:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:52:43.198215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:43.259797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:43.296357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff51fd-c252-42e3-be92-30cae3f8e63b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:43.328334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:43.329842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:43.330159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:43.330528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:43.331223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:43.331528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:43.332962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf093f9-3cfd-4649-9318-5b6ca0f119f1 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:43.344615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 549/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:52:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:52:43.350966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:43.449700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530e565-ef2c-4aee-a2a2-b2aa6c354324 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:43.472447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:43.484047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:43.484632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:43.484944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:43.485213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:43.486414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] RBAC: Adding query filter params (role.id=c722fc4c0c804be2a4eb9cadb419c454, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=ec86be5855254c8abc10fb636add2749) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:43.486717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:43.488137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016a191-0654-469f-8c62-380d9c270820 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:43.493834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 557/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:43 2022] GET /identity/v3/role_assignments?user.id=ec86be5855254c8abc10fb636add2749&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=c722fc4c0c804be2a4eb9cadb419c454 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:44.144130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb72362-2c7e-4c24-8aa7-f9cb3c492ef9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:44.144654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb72362-2c7e-4c24-8aa7-f9cb3c492ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:44.144654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb72362-2c7e-4c24-8aa7-f9cb3c492ef9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:44.146019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 550/1107] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:44.151446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:44.151859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:44.152067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:44.171100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:44.172280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:44.242211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 558/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:44.517142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb72362-2c7e-4c24-8aa7-f9cb3c492ef9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:44.613599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb72362-2c7e-4c24-8aa7-f9cb3c492ef9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:44.636083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:44.647049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e1610-954a-48e5-9d34-ab1f05f314c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:44.647655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e1610-954a-48e5-9d34-ab1f05f314c6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:44.647929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e1610-954a-48e5-9d34-ab1f05f314c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:44.648196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e1610-954a-48e5-9d34-ab1f05f314c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:44.649299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e1610-954a-48e5-9d34-ab1f05f314c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:44.650718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e1610-954a-48e5-9d34-ab1f05f314c6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:44.652244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 551/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:44.658192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:44.755718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd65e7-2225-46f4-a866-0417b7b7c102 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:44.771866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:44.788999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40790bff-8f1d-48de-b0dc-225ff64b0e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:44.789588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790bff-8f1d-48de-b0dc-225ff64b0e53 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:44.789859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790bff-8f1d-48de-b0dc-225ff64b0e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:44.790130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40790bff-8f1d-48de-b0dc-225ff64b0e53 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:44.790762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40790bff-8f1d-48de-b0dc-225ff64b0e53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:44.792187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40790bff-8f1d-48de-b0dc-225ff64b0e53 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:44.811405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 559/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:52:44 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:46.965037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17dd48-f1f4-4b31-a639-dece60d7738d None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:46.965562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17dd48-f1f4-4b31-a639-dece60d7738d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:46.965562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17dd48-f1f4-4b31-a639-dece60d7738d None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:46.966901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 552/1111] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:46.972713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:46.973154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:46.973276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:46.991746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:46.992969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:47.056756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 560/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:47.323715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17dd48-f1f4-4b31-a639-dece60d7738d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:47.421599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17dd48-f1f4-4b31-a639-dece60d7738d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:47.452844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:47.453448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:47.453723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:47.454856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] PATH_INFO: `/v3/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:47.456531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] RBAC: Authorizing `identity:get_role(role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:47.457897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:47.459203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 553/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:47 2022] GET /identity/v3/roles/44957acdbea44978962b0cc8d2e0d069 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:47.465040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:47.561207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e347d1-ee70-4bba-95d4-1519554f2b1c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:47.562670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:47.595928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:47.596475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:47.596776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:47.597051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] PATH_INFO: `/v3/users/6f33102ec75340dba6baed8d49f221bc` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:47.609653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] RBAC: Authorizing `identity:get_user(user_id=6f33102ec75340dba6baed8d49f221bc)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:47.611206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:47.612813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 561/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:47 2022] GET /identity/v3/users/6f33102ec75340dba6baed8d49f221bc => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:47.618547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:47.675366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:47.715050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc79bc-6f25-4423-9ab0-a256205a1f00 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:47.746727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:47.747311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:47.747589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:47.747858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:47.811466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:47.812917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:47.867515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:47.907822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 554/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:47.913435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:48.009057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687b679-3d7d-4766-8b67-3b37c4073bea None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:48.040814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b22faa-aef2-443c-94be-3b463db61049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:48.041414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b22faa-aef2-443c-94be-3b463db61049 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:48.041689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b22faa-aef2-443c-94be-3b463db61049 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:48.041957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b22faa-aef2-443c-94be-3b463db61049 None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:48.043039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b22faa-aef2-443c-94be-3b463db61049 None admin] RBAC: Authorizing `identity:get_project(project_id=b34ded98138f45549fbde2161bfa2164)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:48.044444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b22faa-aef2-443c-94be-3b463db61049 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:48.046072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 562/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:52:47 2022] GET /identity/v3/projects/b34ded98138f45549fbde2161bfa2164 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:48.052664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:48.146981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8f355-b3b5-4a83-9bbf-6b1304aaf550 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:48.179443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:48.180026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:48.180302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:48.180572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:48.181316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] RBAC: Adding query filter params (role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=b34ded98138f45549fbde2161bfa2164, user.id=6f33102ec75340dba6baed8d49f221bc) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:48.181617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:48.183022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56de6f-6553-45d8-91e3-643655aada4f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:48.188091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 555/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:48 2022] GET /identity/v3/role_assignments?user.id=6f33102ec75340dba6baed8d49f221bc&scope.project.id=b34ded98138f45549fbde2161bfa2164&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:50.287629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eadf65-ecd8-4d3c-9407-b70bc1351ae6 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:50.288174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eadf65-ecd8-4d3c-9407-b70bc1351ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:50.288174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eadf65-ecd8-4d3c-9407-b70bc1351ae6 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:50.289537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 563/1118] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:50.294824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:50.295233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:50.295366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:50.313163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:50.314349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:50.331246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:50.378790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 556/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:50.650607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eadf65-ecd8-4d3c-9407-b70bc1351ae6 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:50.678418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:50.751893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eadf65-ecd8-4d3c-9407-b70bc1351ae6 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:50.786287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:50.786908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:50.787186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:50.787468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] PATH_INFO: `/v3/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:50.788875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] RBAC: Authorizing `identity:get_role(role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:50.790250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:50.791581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 564/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:50 2022] GET /identity/v3/roles/44957acdbea44978962b0cc8d2e0d069 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:50.797345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:50.890945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c093724b-0bcb-4c4e-9a72-d754b6c2b500 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:50.923572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:50.924127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:50.924402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:50.924690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] PATH_INFO: `/v3/users/6f33102ec75340dba6baed8d49f221bc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:50.925791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] RBAC: Authorizing `identity:get_user(user_id=6f33102ec75340dba6baed8d49f221bc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:50.927202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:50.928781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 557/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:50 2022] GET /identity/v3/users/6f33102ec75340dba6baed8d49f221bc => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:50.934433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:51.024824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:51.031060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3b0d9-9ad4-4dac-a269-bb5e5cda52ad None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:51.064575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb58417-b12f-4d0b-ab17-949e9a0828a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:51.065239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb58417-b12f-4d0b-ab17-949e9a0828a1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.065515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb58417-b12f-4d0b-ab17-949e9a0828a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.065786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb58417-b12f-4d0b-ab17-949e9a0828a1 None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.066887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb58417-b12f-4d0b-ab17-949e9a0828a1 None admin] RBAC: Authorizing `identity:get_project(project_id=b34ded98138f45549fbde2161bfa2164)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:51.068315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb58417-b12f-4d0b-ab17-949e9a0828a1 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:51.069967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 565/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:52:50 2022] GET /identity/v3/projects/b34ded98138f45549fbde2161bfa2164 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:51.075673 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:51.099209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:51.181386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788061-3ba2-485c-b233-d6af6dafaa59 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:51.213622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba55cb-ffb1-4fc7-811e-a349933e0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:51.214259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba55cb-ffb1-4fc7-811e-a349933e0540 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:51.214533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba55cb-ffb1-4fc7-811e-a349933e0540 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:51.214802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba55cb-ffb1-4fc7-811e-a349933e0540 None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164/users/6f33102ec75340dba6baed8d49f221bc/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:51.216898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba55cb-ffb1-4fc7-811e-a349933e0540 None admin] RBAC: Authorizing `identity:create_grant(project_id=b34ded98138f45549fbde2161bfa2164, user_id=6f33102ec75340dba6baed8d49f221bc, role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:51.218304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba55cb-ffb1-4fc7-811e-a349933e0540 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:51.230751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 558/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:52:51 2022] PUT /identity/v3/projects/b34ded98138f45549fbde2161bfa2164/users/6f33102ec75340dba6baed8d49f221bc/roles/44957acdbea44978962b0cc8d2e0d069 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:52:53.349352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0e563-0630-472d-9602-4e4f33b82fdb None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:53.349886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0e563-0630-472d-9602-4e4f33b82fdb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:53.349975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0e563-0630-472d-9602-4e4f33b82fdb None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:53.351236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 566/1124] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:53.356386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:53.356761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:53.356968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:53.375037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:53.376600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:53.442305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 559/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:53.709919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0e563-0630-472d-9602-4e4f33b82fdb None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:53.832092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0e563-0630-472d-9602-4e4f33b82fdb None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:53.863615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:53.864224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:53.864496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:53.864795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] PATH_INFO: `/v3/roles/44957acdbea44978962b0cc8d2e0d069` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:53.866150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] RBAC: Authorizing `identity:get_role(role_id=44957acdbea44978962b0cc8d2e0d069)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:53.867505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:53.868838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 567/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:53 2022] GET /identity/v3/roles/44957acdbea44978962b0cc8d2e0d069 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:53.874681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:53.888788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:53.971536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f129cd-7e56-4c1c-948e-4db6bb534465 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:54.004354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:54.005000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:54.005276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:54.005545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] PATH_INFO: `/v3/users/6f33102ec75340dba6baed8d49f221bc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:54.006641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] RBAC: Authorizing `identity:get_user(user_id=6f33102ec75340dba6baed8d49f221bc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:54.008051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:54.009625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 560/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:52:53 2022] GET /identity/v3/users/6f33102ec75340dba6baed8d49f221bc => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:54.015444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:54.045516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:54.125262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc7845-487d-46b1-804d-c315b4af242a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:54.159193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:54.159781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:54.160054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:54.160322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:54.225829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:54.227224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:54.256765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:54.324066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 568/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:52:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:52:54.330234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:54.407494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:54.426655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d2a6-c609-4859-b519-a9b9313947b7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:54.458673 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75ab86-e683-4f84-87fc-14368c80462d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:54.459279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75ab86-e683-4f84-87fc-14368c80462d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:54.459555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75ab86-e683-4f84-87fc-14368c80462d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:54.459851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75ab86-e683-4f84-87fc-14368c80462d None admin] PATH_INFO: `/v3/projects/b34ded98138f45549fbde2161bfa2164` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:54.460974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75ab86-e683-4f84-87fc-14368c80462d None admin] RBAC: Authorizing `identity:get_project(project_id=b34ded98138f45549fbde2161bfa2164)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:54.462388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75ab86-e683-4f84-87fc-14368c80462d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:54.464003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 561/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:52:54 2022] GET /identity/v3/projects/b34ded98138f45549fbde2161bfa2164 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:54.470994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:54.568307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea5cf0-7acd-4cf9-b2ec-9a89bd05518c None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:54.597460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:54.602710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:54.603303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:54.603581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:54.603856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:54.604578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] RBAC: Adding query filter params (role.id=44957acdbea44978962b0cc8d2e0d069, scope.project.id=b34ded98138f45549fbde2161bfa2164, user.id=6f33102ec75340dba6baed8d49f221bc) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:52:54.604913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:54.606346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344e654-b32e-4b02-a98b-6f99daad6ffb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:54.612911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 569/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:52:54 2022] GET /identity/v3/role_assignments?user.id=6f33102ec75340dba6baed8d49f221bc&scope.project.id=b34ded98138f45549fbde2161bfa2164&role.id=44957acdbea44978962b0cc8d2e0d069 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:56.809506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54730-ddc9-4e2a-a26a-c4c94beacb31 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:56.810039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54730-ddc9-4e2a-a26a-c4c94beacb31 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:56.810039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54730-ddc9-4e2a-a26a-c4c94beacb31 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:56.811370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 562/1131] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:56.816853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:56.817254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:56.817402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:56.836933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:56.838135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:56.906348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 570/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:52:57.182485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54730-ddc9-4e2a-a26a-c4c94beacb31 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.283021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54730-ddc9-4e2a-a26a-c4c94beacb31 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.319177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e03030-008b-40a6-8308-c981560c2c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.320209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e03030-008b-40a6-8308-c981560c2c7c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.320698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e03030-008b-40a6-8308-c981560c2c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.321187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e03030-008b-40a6-8308-c981560c2c7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.322908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e03030-008b-40a6-8308-c981560c2c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.325483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e03030-008b-40a6-8308-c981560c2c7c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.328341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 563/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:52:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:52:57.334946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:52:57.432713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30008f-7cac-4714-861c-27afe958d114 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:52:57.458172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:57.467384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91738e2-c0f0-42f7-a414-8a0b51200cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:52:57.467984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91738e2-c0f0-42f7-a414-8a0b51200cc9 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:57.468260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91738e2-c0f0-42f7-a414-8a0b51200cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:57.468532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91738e2-c0f0-42f7-a414-8a0b51200cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:57.469310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91738e2-c0f0-42f7-a414-8a0b51200cc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:52:57.470863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91738e2-c0f0-42f7-a414-8a0b51200cc9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:52:57.496889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 571/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:52:57 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:52:59.707107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f285be-f946-4564-a24d-67e8825cdf40 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:59.707724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f285be-f946-4564-a24d-67e8825cdf40 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:59.708065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f285be-f946-4564-a24d-67e8825cdf40 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:59.709596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 564/1135] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:52:59.715060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:52:59.715508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:52:59.715600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:52:59.734118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:52:59.735934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:52:59.801214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 572/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:52:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:00.073013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f285be-f946-4564-a24d-67e8825cdf40 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:00.091322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:00.174314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f285be-f946-4564-a24d-67e8825cdf40 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:00.217296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310f2a2-1247-4241-bf84-89583c444fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:00.217923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f2a2-1247-4241-bf84-89583c444fce None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:00.218201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f2a2-1247-4241-bf84-89583c444fce None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:00.218473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f2a2-1247-4241-bf84-89583c444fce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:00.220112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310f2a2-1247-4241-bf84-89583c444fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:00.221559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310f2a2-1247-4241-bf84-89583c444fce None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:00.223131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 565/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:53:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:00.228909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:00.325333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed473a14-31bf-486c-8a64-336557cb6df9 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:00.334913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:00.361135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f63282-a345-46f8-a137-48351234c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:00.361722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f63282-a345-46f8-a137-48351234c28f None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:00.361998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f63282-a345-46f8-a137-48351234c28f None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:00.362265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f63282-a345-46f8-a137-48351234c28f None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:00.362911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f63282-a345-46f8-a137-48351234c28f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:00.364339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f63282-a345-46f8-a137-48351234c28f None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:00.384968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 573/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:53:00 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:02.595678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c3f0-a7a7-44e3-94e3-5c135594530c None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:02.596194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c3f0-a7a7-44e3-94e3-5c135594530c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:02.596194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b6c3f0-a7a7-44e3-94e3-5c135594530c None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:02.597585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 566/1139] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:02.602829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:02.603250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:02.603378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:02.621758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:02.622944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:02.687053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 574/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:02.954908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6c3f0-a7a7-44e3-94e3-5c135594530c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:03.037293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:03.053714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b6c3f0-a7a7-44e3-94e3-5c135594530c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:03.086431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:03.087038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:03.087313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:03.087579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:03.098023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:53:03.102818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:03.103891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:53:03.105153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 567/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:53:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:03.109957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:03.199943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:03.208651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb6535-0370-4a50-b90f-c4332fd2993a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:03.241769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:03.242359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:03.242638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:03.242912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:03.243570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:03.243870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:03.245279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:03.253063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 575/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:53:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:03.258991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:03.355463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addf79d-0328-4b7e-b2e7-4bf95348503d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:03.389063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:03.389680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:03.389956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:03.390225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] PATH_INFO: `/v3/users/76696ab3c53941e9ae73a7ba44f1fcdb` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:03.403782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] RBAC: Authorizing `identity:get_user(user_id=76696ab3c53941e9ae73a7ba44f1fcdb)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:03.405256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:03.406905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 568/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:53:03 2022] GET /identity/v3/users/76696ab3c53941e9ae73a7ba44f1fcdb => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:03.412661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:03.507976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2a7b3-3fe2-45e7-b52c-13ef69aa706b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:03.540192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:03.540827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:03.541100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:03.541366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:03.565024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:03.607190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:03.608638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:03.702636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 576/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:53:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:53:03.709623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:03.712414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:03.805969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0023808-36fc-4b3a-b7c7-04ba852fc74f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:03.838622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592588a-98d7-4ce6-afaf-b083c0236069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:03.839229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592588a-98d7-4ce6-afaf-b083c0236069 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:03.839505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592588a-98d7-4ce6-afaf-b083c0236069 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:03.839777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592588a-98d7-4ce6-afaf-b083c0236069 None admin] PATH_INFO: `/v3/projects/51a2c59abee0496cb649b84ce316314c` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:03.840894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592588a-98d7-4ce6-afaf-b083c0236069 None admin] RBAC: Authorizing `identity:get_project(project_id=51a2c59abee0496cb649b84ce316314c)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:03.842312 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592588a-98d7-4ce6-afaf-b083c0236069 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:03.844424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 569/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:53:03 2022] GET /identity/v3/projects/51a2c59abee0496cb649b84ce316314c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:03.850071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:03.903966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:03.947109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd738-9477-4b11-98b8-a71e745fedea None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:03.980584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:03.981203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:03.981482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:03.981751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:03.982469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=51a2c59abee0496cb649b84ce316314c, user.id=76696ab3c53941e9ae73a7ba44f1fcdb) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:03.982773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:03.984198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc740b-39d5-4bb2-b99e-dae6502e0b9b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:03.990132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 577/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:53:03 2022] GET /identity/v3/role_assignments?user.id=76696ab3c53941e9ae73a7ba44f1fcdb&scope.project.id=51a2c59abee0496cb649b84ce316314c&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:06.113156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52934242-2dfc-4717-a4fa-bdd206726c85 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:06.113825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52934242-2dfc-4717-a4fa-bdd206726c85 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:06.113825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52934242-2dfc-4717-a4fa-bdd206726c85 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:06.115269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 570/1147] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:06.120648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:06.121056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:06.121223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:06.139493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:06.140706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:06.205994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 578/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:06.475952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52934242-2dfc-4717-a4fa-bdd206726c85 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:06.544870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:06.575823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52934242-2dfc-4717-a4fa-bdd206726c85 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:06.608183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:06.608815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:06.609090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:06.609357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:06.618961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:53:06.623732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:06.624927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:53:06.626678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 571/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:06.631501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:06.678681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:06.727014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93a851-b3e1-430d-a92b-d31bbeda7cd9 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:06.759729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:06.760309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:06.760580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:06.760878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:06.761523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:06.761814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:06.763169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:06.771537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 579/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:06.777361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:06.873869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8808a4e-a635-42cf-b211-0d70efee6bb1 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:06.901469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:06.908094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:06.908721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:06.909002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:06.909274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] PATH_INFO: `/v3/users/76696ab3c53941e9ae73a7ba44f1fcdb` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:06.910367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] RBAC: Authorizing `identity:get_user(user_id=76696ab3c53941e9ae73a7ba44f1fcdb)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:06.911782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:06.913377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 572/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:53:06 2022] GET /identity/v3/users/76696ab3c53941e9ae73a7ba44f1fcdb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:06.919026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:07.014898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebb079-5159-4b2e-81a9-75cf4bea0cbd None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:07.027381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:07.048939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98f380-d188-4572-aa9f-c6745f9ed303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:07.049552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f380-d188-4572-aa9f-c6745f9ed303 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:07.049826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f380-d188-4572-aa9f-c6745f9ed303 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:07.050098 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98f380-d188-4572-aa9f-c6745f9ed303 None admin] PATH_INFO: `/v3/projects/51a2c59abee0496cb649b84ce316314c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:07.051187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98f380-d188-4572-aa9f-c6745f9ed303 None admin] RBAC: Authorizing `identity:get_project(project_id=51a2c59abee0496cb649b84ce316314c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:07.052617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98f380-d188-4572-aa9f-c6745f9ed303 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:07.054289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 580/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:53:06 2022] GET /identity/v3/projects/51a2c59abee0496cb649b84ce316314c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:07.060135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:07.156640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f5fc4-71ba-4de5-9613-7b75dd793274 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:07.189938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384862b-fc96-442d-a2b0-2d0f84957724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:07.190575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384862b-fc96-442d-a2b0-2d0f84957724 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:07.190850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384862b-fc96-442d-a2b0-2d0f84957724 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:07.191123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384862b-fc96-442d-a2b0-2d0f84957724 None admin] PATH_INFO: `/v3/projects/51a2c59abee0496cb649b84ce316314c/users/76696ab3c53941e9ae73a7ba44f1fcdb/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:07.193027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384862b-fc96-442d-a2b0-2d0f84957724 None admin] RBAC: Authorizing `identity:create_grant(project_id=51a2c59abee0496cb649b84ce316314c, user_id=76696ab3c53941e9ae73a7ba44f1fcdb, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:07.194423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384862b-fc96-442d-a2b0-2d0f84957724 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:07.207558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 573/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:53:07 2022] PUT /identity/v3/projects/51a2c59abee0496cb649b84ce316314c/users/76696ab3c53941e9ae73a7ba44f1fcdb/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:09.336395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14cb6-47e9-4419-94d2-f2958f4064fb None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:09.337000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14cb6-47e9-4419-94d2-f2958f4064fb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:09.337000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14cb6-47e9-4419-94d2-f2958f4064fb None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:09.338321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 581/1154] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:09.344162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:09.345011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:09.345304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:09.363906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:09.365113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:09.424761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:09.438612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 574/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:09.708389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14cb6-47e9-4419-94d2-f2958f4064fb None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:09.807122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14cb6-47e9-4419-94d2-f2958f4064fb None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:09.839490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:09.840101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:09.840381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:09.840658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:09.851445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:53:09.856184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:09.857290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:53:09.858150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 582/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:53:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:09.863565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:09.961417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ab89c-4be1-4419-850e-feda7ffae009 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:09.994630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:09.995217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:09.995494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:09.995766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:09.996420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:09.996742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:09.998099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:10.006135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 575/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:53:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:10.011941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:10.052640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:10.109091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269692b2-24c1-4f37-8a54-b7f8950cc382 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:10.141988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:10.142597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:10.142873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:10.143143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] PATH_INFO: `/v3/users/76696ab3c53941e9ae73a7ba44f1fcdb` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:10.144235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] RBAC: Authorizing `identity:get_user(user_id=76696ab3c53941e9ae73a7ba44f1fcdb)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:10.145687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:10.147258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 583/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:53:10 2022] GET /identity/v3/users/76696ab3c53941e9ae73a7ba44f1fcdb => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:10.153020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:10.200478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:10.250455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcff63-1b5c-49ef-a98c-88b4fb18d239 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:10.283248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:10.283840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:10.284114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:10.284379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:10.348529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:10.349989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:10.396699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:10.444803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 576/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:53:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:53:10.450666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:10.551833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f446c-39bd-422d-b51a-0a0dd108ef99 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:10.560090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:10.584476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1ba84-a270-4ff0-befe-b616ead0e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:10.585110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1ba84-a270-4ff0-befe-b616ead0e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:10.585381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1ba84-a270-4ff0-befe-b616ead0e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:10.585646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1ba84-a270-4ff0-befe-b616ead0e9b8 None admin] PATH_INFO: `/v3/projects/51a2c59abee0496cb649b84ce316314c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:10.586752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1ba84-a270-4ff0-befe-b616ead0e9b8 None admin] RBAC: Authorizing `identity:get_project(project_id=51a2c59abee0496cb649b84ce316314c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:10.588172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1ba84-a270-4ff0-befe-b616ead0e9b8 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:10.589805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 584/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:53:10 2022] GET /identity/v3/projects/51a2c59abee0496cb649b84ce316314c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:10.596176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:10.694321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd15a49-62a5-4f3e-bb2a-f55ea48bae5c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:10.726228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:10.726819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:10.727097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:10.727368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:10.728084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=51a2c59abee0496cb649b84ce316314c, user.id=76696ab3c53941e9ae73a7ba44f1fcdb) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:10.728387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:10.729826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c466997-293b-49f0-8d6b-2b1de31ab97b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:10.735911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 577/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:53:10 2022] GET /identity/v3/role_assignments?user.id=76696ab3c53941e9ae73a7ba44f1fcdb&scope.project.id=51a2c59abee0496cb649b84ce316314c&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:12.882000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bd072-8555-4fd2-a93f-625f6334bf45 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:12.882549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bd072-8555-4fd2-a93f-625f6334bf45 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:12.882549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bd072-8555-4fd2-a93f-625f6334bf45 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:12.883862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 585/1162] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:12.888918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:12.889348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:12.889461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:12.908396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:12.909604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:12.918474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:12.973824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 578/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:13.241175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bd072-8555-4fd2-a93f-625f6334bf45 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.339615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bd072-8555-4fd2-a93f-625f6334bf45 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.371523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.372117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.372394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.372667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.416987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:13.439514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.440985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.535104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 586/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:53:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:53:13.541567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.637918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c280b8-9417-43ad-90b1-efb9a9caf516 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.669468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.670076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.670350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.670618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.677495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.678916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-fefe2d25-5edc-41f7-a6a1-ca3462c5937d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:13.690502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:13.691769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:53:13.693013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 579/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:53:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:13.697435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:13.770202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:13.792207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce28e8-ecc3-46e6-bd81-f73884abb570 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:13.825203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:13.825782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:13.826053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:13.826322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:13.826944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:13.827240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:13.828990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8e75b-6500-4614-9f4d-7b65fd116f99 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:13.835827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 587/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Sat Sep 17 20:53:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:15.931999 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5da3f-7c11-4c02-8cf5-6cfc4f148cbf None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:15.932520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5da3f-7c11-4c02-8cf5-6cfc4f148cbf None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:15.932520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5da3f-7c11-4c02-8cf5-6cfc4f148cbf None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:15.933913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 580/1167] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:15.939953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2262e-878d-46be-aab2-2fe929fec1ea None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:15.940330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2262e-878d-46be-aab2-2fe929fec1ea None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:15.940493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2262e-878d-46be-aab2-2fe929fec1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:15.958391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-35a2262e-878d-46be-aab2-2fe929fec1ea None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:15.959558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-35a2262e-878d-46be-aab2-2fe929fec1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:16.023069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 588/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:53:16.289938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5da3f-7c11-4c02-8cf5-6cfc4f148cbf None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:16.293755 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:16.388082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5da3f-7c11-4c02-8cf5-6cfc4f148cbf None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:16.420822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d386760-5b9f-46b6-8dad-9e5b7064b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:16.421414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d386760-5b9f-46b6-8dad-9e5b7064b95d None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:16.422376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d386760-5b9f-46b6-8dad-9e5b7064b95d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:16.422651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d386760-5b9f-46b6-8dad-9e5b7064b95d None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:16.423316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d386760-5b9f-46b6-8dad-9e5b7064b95d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:16.424693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d386760-5b9f-46b6-8dad-9e5b7064b95d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:16.439750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 581/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Sat Sep 17 20:53:16 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:18.657649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a24544-8df5-44bc-867a-26ff02da42c0 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:18.658145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a24544-8df5-44bc-867a-26ff02da42c0 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:18.658145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a24544-8df5-44bc-867a-26ff02da42c0 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:18.659573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 589/1170] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:18.664906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:18.665336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:18.665462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:18.683817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:18.685048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:18.741683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:18.751209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 582/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:19.019728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a24544-8df5-44bc-867a-26ff02da42c0 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:19.088251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:19.119431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a24544-8df5-44bc-867a-26ff02da42c0 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:19.153240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abea418-11db-4781-b360-b1d9c80e629c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:19.153852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abea418-11db-4781-b360-b1d9c80e629c None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:19.154124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abea418-11db-4781-b360-b1d9c80e629c None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:19.154392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abea418-11db-4781-b360-b1d9c80e629c None admin] PATH_INFO: `/v3/domains/dfb99a342d204ea1bef5b522620f8a6b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:19.164554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abea418-11db-4781-b360-b1d9c80e629c None admin] RBAC: Authorizing `identity:get_domain(domain_id=dfb99a342d204ea1bef5b522620f8a6b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:19.166016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abea418-11db-4781-b360-b1d9c80e629c None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:19.167581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 590/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Sat Sep 17 20:53:19 2022] GET /identity/v3/domains/dfb99a342d204ea1bef5b522620f8a6b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:19.173667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:19.268494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358a97-c677-43a3-be05-35d2a9f3706e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:19.300575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fad1b2-c054-4e2c-94ee-25715b3a652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:19.301141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fad1b2-c054-4e2c-94ee-25715b3a652e None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:19.301421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fad1b2-c054-4e2c-94ee-25715b3a652e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:19.301695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fad1b2-c054-4e2c-94ee-25715b3a652e None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:19.302345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fad1b2-c054-4e2c-94ee-25715b3a652e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:19.303757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fad1b2-c054-4e2c-94ee-25715b3a652e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:19.330501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 583/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:53:19 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:53:21.508517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660369f-0526-4438-8f81-5495b0d8e868 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:21.509139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660369f-0526-4438-8f81-5495b0d8e868 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:21.509139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660369f-0526-4438-8f81-5495b0d8e868 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:21.510405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 591/1174] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:21.515782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:21.516246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:21.516353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:21.534609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:21.535777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:21.575815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:21.605642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 584/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:21.884380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660369f-0526-4438-8f81-5495b0d8e868 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:21.985683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5660369f-0526-4438-8f81-5495b0d8e868 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:22.012731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:22.019425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cb6aa-1a4e-442f-9c90-e2cf78a580bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:22.020043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cb6aa-1a4e-442f-9c90-e2cf78a580bb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:22.020319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cb6aa-1a4e-442f-9c90-e2cf78a580bb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:22.020590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cb6aa-1a4e-442f-9c90-e2cf78a580bb None admin] PATH_INFO: `/v3/domains/dfb99a342d204ea1bef5b522620f8a6b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:22.021709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cb6aa-1a4e-442f-9c90-e2cf78a580bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=dfb99a342d204ea1bef5b522620f8a6b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:22.023140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cb6aa-1a4e-442f-9c90-e2cf78a580bb None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:22.024742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 592/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Sat Sep 17 20:53:21 2022] GET /identity/v3/domains/dfb99a342d204ea1bef5b522620f8a6b => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:22.030525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:22.125705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fe3e-2d9f-406c-a7ec-0307d603c212 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:22.157973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b001a0-e164-4234-aaca-32fed7b774ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:22.158500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b001a0-e164-4234-aaca-32fed7b774ed None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:22.158776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b001a0-e164-4234-aaca-32fed7b774ed None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:22.159045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b001a0-e164-4234-aaca-32fed7b774ed None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:22.159692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b001a0-e164-4234-aaca-32fed7b774ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:22.161137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b001a0-e164-4234-aaca-32fed7b774ed None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:22.182152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 585/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:53:22 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 20:53:24.364001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47e8b2-5f80-428b-86bb-9848d1878790 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:24.364568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47e8b2-5f80-428b-86bb-9848d1878790 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:24.364568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47e8b2-5f80-428b-86bb-9848d1878790 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:24.365899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 593/1178] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:24.371041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:24.371452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:24.371583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:24.389701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:24.391917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:24.416465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:24.456684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 586/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:24.725449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47e8b2-5f80-428b-86bb-9848d1878790 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:24.824781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47e8b2-5f80-428b-86bb-9848d1878790 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:24.856668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:24.857304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:24.857581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:24.857853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:24.868587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:53:24.873311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:24.875139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:53:24.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 594/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:53:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:24.881040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:24.980467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac38d-4b7c-48e2-8d27-bac475f0df22 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:25.012518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:25.013148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.013425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.013694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.014346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:25.014641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:25.016539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:25.024189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 587/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:53:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:25.030094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:25.086088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:25.128060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99af53d-584b-4b4f-b39f-33fb0c865a45 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:25.160129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85154383-c121-4628-b358-993f80e52592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:25.160766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85154383-c121-4628-b358-993f80e52592 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.161042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85154383-c121-4628-b358-993f80e52592 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.161310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85154383-c121-4628-b358-993f80e52592 None admin] PATH_INFO: `/v3/users/a3c6ac4e209a4e008d62f31ca9f60ad2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.173833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85154383-c121-4628-b358-993f80e52592 None admin] RBAC: Authorizing `identity:get_user(user_id=a3c6ac4e209a4e008d62f31ca9f60ad2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:25.175265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85154383-c121-4628-b358-993f80e52592 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:25.176863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 595/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:53:25 2022] GET /identity/v3/users/a3c6ac4e209a4e008d62f31ca9f60ad2 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:25.182593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:25.208564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:25.281032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc8293-7ee9-483d-9a2b-e43cf0250b58 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:25.313255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:25.313842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.314116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.314385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.378362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:25.379797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:25.403306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:25.480860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 588/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:53:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:53:25.487548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85154383-c121-4628-b358-993f80e52592 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:25.584545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85154383-c121-4628-b358-993f80e52592 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:25.600225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:25.618328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc72c7-650e-4195-8276-7a21b7fbf1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:25.619010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc72c7-650e-4195-8276-7a21b7fbf1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.619293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc72c7-650e-4195-8276-7a21b7fbf1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.619570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc72c7-650e-4195-8276-7a21b7fbf1b6 None admin] PATH_INFO: `/v3/projects/49a64bb281a849198a970d947f6fee71` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.620699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc72c7-650e-4195-8276-7a21b7fbf1b6 None admin] RBAC: Authorizing `identity:get_project(project_id=49a64bb281a849198a970d947f6fee71)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:25.622137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc72c7-650e-4195-8276-7a21b7fbf1b6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:25.623752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 596/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:53:25 2022] GET /identity/v3/projects/49a64bb281a849198a970d947f6fee71 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:25.630444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:25.723509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3ee76-f81f-4e76-966a-444cedf01a6d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:25.746270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:25.757319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:25.757899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:25.758172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:25.758438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:25.759152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=49a64bb281a849198a970d947f6fee71, user.id=a3c6ac4e209a4e008d62f31ca9f60ad2) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:25.759449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:25.760867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce28e83-828e-42bb-b553-6e640fe1c4e7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:25.766223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 589/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:53:25 2022] GET /identity/v3/role_assignments?user.id=a3c6ac4e209a4e008d62f31ca9f60ad2&scope.project.id=49a64bb281a849198a970d947f6fee71&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:27.877058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e772-2f13-4bd1-b7ed-f2b590b5dfe5 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:27.877584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e772-2f13-4bd1-b7ed-f2b590b5dfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:27.877584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e772-2f13-4bd1-b7ed-f2b590b5dfe5 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:27.878938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 597/1186] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:27.884354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:27.884814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:27.884935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:27.904151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:27.905422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:27.972232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 590/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:28.245145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e772-2f13-4bd1-b7ed-f2b590b5dfe5 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:28.344047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e772-2f13-4bd1-b7ed-f2b590b5dfe5 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:28.377745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:28.378358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:28.378636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:28.378901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:28.388713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:53:28.393940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:28.394990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:53:28.396091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 598/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:28.401101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:28.494971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12144b74-e58e-4e24-8474-29bce8a62795 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:28.511858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:28.528535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:28.529161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:28.529436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:28.529701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:28.530426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:28.530668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:28.532115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:28.539759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 591/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:28.545614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:28.590798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:28.643837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2675d1-db76-4aae-b5ef-3eab4aaacc01 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:28.676476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:28.677117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:28.677395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:28.677663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] PATH_INFO: `/v3/users/a3c6ac4e209a4e008d62f31ca9f60ad2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:28.678755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] RBAC: Authorizing `identity:get_user(user_id=a3c6ac4e209a4e008d62f31ca9f60ad2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:28.680176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:28.681766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 599/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:53:28 2022] GET /identity/v3/users/a3c6ac4e209a4e008d62f31ca9f60ad2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:28.688253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:28.783421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f38d90-b3ba-4a7e-8e9a-d829fa6a99b5 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:28.815945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51540ea6-8994-42d5-ac28-6e538a4618f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:28.816549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540ea6-8994-42d5-ac28-6e538a4618f5 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:28.816854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540ea6-8994-42d5-ac28-6e538a4618f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:28.817126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540ea6-8994-42d5-ac28-6e538a4618f5 None admin] PATH_INFO: `/v3/projects/49a64bb281a849198a970d947f6fee71` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:28.818227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51540ea6-8994-42d5-ac28-6e538a4618f5 None admin] RBAC: Authorizing `identity:get_project(project_id=49a64bb281a849198a970d947f6fee71)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:28.819640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51540ea6-8994-42d5-ac28-6e538a4618f5 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:28.821265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 592/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:53:28 2022] GET /identity/v3/projects/49a64bb281a849198a970d947f6fee71 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:28.827053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:28.925656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96efd48-374f-4de9-a285-f8fc8ae4120a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:28.940077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:28.961867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56e7a0-bef3-44d1-a3e5-752fdbd99ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:28.962512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56e7a0-bef3-44d1-a3e5-752fdbd99ba7 None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:28.962790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56e7a0-bef3-44d1-a3e5-752fdbd99ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:28.963064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56e7a0-bef3-44d1-a3e5-752fdbd99ba7 None admin] PATH_INFO: `/v3/projects/49a64bb281a849198a970d947f6fee71/users/a3c6ac4e209a4e008d62f31ca9f60ad2/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:28.964970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56e7a0-bef3-44d1-a3e5-752fdbd99ba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=49a64bb281a849198a970d947f6fee71, user_id=a3c6ac4e209a4e008d62f31ca9f60ad2, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:28.966393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56e7a0-bef3-44d1-a3e5-752fdbd99ba7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:28.979685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 600/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:53:28 2022] PUT /identity/v3/projects/49a64bb281a849198a970d947f6fee71/users/a3c6ac4e209a4e008d62f31ca9f60ad2/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:53:31.101555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c2ff2-8305-4dfe-9781-15ace22cbcad None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:31.102081 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c2ff2-8305-4dfe-9781-15ace22cbcad None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:31.102197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c2ff2-8305-4dfe-9781-15ace22cbcad None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:31.103440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 593/1193] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:53:31.108613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:31.109086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:31.109263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:31.128373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:53:31.129596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:53:31.193034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 601/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:53:31.463233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c2ff2-8305-4dfe-9781-15ace22cbcad None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:31.511465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:31.566063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c2ff2-8305-4dfe-9781-15ace22cbcad None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:31.597808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:31.598415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:31.598689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:31.598957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:31.608997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:53:31.613724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:53:31.614807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:53:31.615803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 594/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:53:31.620893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:31.716563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a756-b989-4bc1-9e3f-242df5f269eb None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:31.718159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:31.751800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:31.752390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:31.752663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:31.752954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:31.754144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:31.754437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:31.755789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:31.763526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 602/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:53:31.769476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:31.864574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c850fcb-c765-41fb-be38-54002612ea4d None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:31.873919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:31.897675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:31.898280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:31.898552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:31.898818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] PATH_INFO: `/v3/users/a3c6ac4e209a4e008d62f31ca9f60ad2` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:31.899903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] RBAC: Authorizing `identity:get_user(user_id=a3c6ac4e209a4e008d62f31ca9f60ad2)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:31.901342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:31.902874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 595/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Sat Sep 17 20:53:31 2022] GET /identity/v3/users/a3c6ac4e209a4e008d62f31ca9f60ad2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:31.909243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:32.005580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7fa8c-8f91-411b-ae75-d4210ae3ab34 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:32.037656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:32.038247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:32.038520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:32.038792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:32.068135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:32.104154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:32.105647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:32.199569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 603/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:53:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:53:32.205846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:32.300651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372560d-dbbf-405c-be0f-0c11797c9290 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:32.333683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a01f17-3f7c-45d9-b727-c18572a147ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:32.334288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a01f17-3f7c-45d9-b727-c18572a147ef None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:32.334564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a01f17-3f7c-45d9-b727-c18572a147ef None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:32.334832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a01f17-3f7c-45d9-b727-c18572a147ef None admin] PATH_INFO: `/v3/projects/49a64bb281a849198a970d947f6fee71` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:32.335971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a01f17-3f7c-45d9-b727-c18572a147ef None admin] RBAC: Authorizing `identity:get_project(project_id=49a64bb281a849198a970d947f6fee71)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:32.337601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a01f17-3f7c-45d9-b727-c18572a147ef None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:32.339261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 596/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Sat Sep 17 20:53:32 2022] GET /identity/v3/projects/49a64bb281a849198a970d947f6fee71 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:32.345930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:53:32.406831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:32.442406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01a9c6-bf8c-484f-a85b-849379bb8bd7 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:53:32.475784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:53:32.476368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:53:32.476658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:53:32.476937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:53:32.477641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=49a64bb281a849198a970d947f6fee71, user.id=a3c6ac4e209a4e008d62f31ca9f60ad2) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:53:32.477938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:53:32.479356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad84e887-68c7-4311-b313-1cfd1dfa6288 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:53:32.484974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 604/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:53:32 2022] GET /identity/v3/role_assignments?user.id=a3c6ac4e209a4e008d62f31ca9f60ad2&scope.project.id=49a64bb281a849198a970d947f6fee71&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:53:42.329677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=78 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:42.482347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=36 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:28.394724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666cdbf-0f34-4d99-bd64-b06293397c99 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:28.394724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666cdbf-0f34-4d99-bd64-b06293397c99 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:28.394724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666cdbf-0f34-4d99-bd64-b06293397c99 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:28.395910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 597/1201] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:28.402180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:28.402686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:28.403428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:28.427619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:28.429293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:28.529588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 605/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:54:28.838605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666cdbf-0f34-4d99-bd64-b06293397c99 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:28.977032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666cdbf-0f34-4d99-bd64-b06293397c99 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:29.015117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:29.016058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:29.016419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:29.016747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:29.028439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:29.030035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:29.049779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:29.050928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:29.051902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:29.051902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:29.051902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:29.051902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 598/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:29.058271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:29.173234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf7f5c-18e1-43d8-bb28-7ab82f458016 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:29.206448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:29.213730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:29.214422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:29.214615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:29.214905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:29.215854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:29.216364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:29.218155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50fabb-9b4a-4d8c-a5b7-5084d10f09d1 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:29.232981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 606/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:29.239815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:29.310935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:29.386198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c70052-a823-4a67-bc36-1109022fd0e2 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:29.431534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33ab12-da8c-4cd7-80da-29c98cf523bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:29.432500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33ab12-da8c-4cd7-80da-29c98cf523bd None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:29.432624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33ab12-da8c-4cd7-80da-29c98cf523bd None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:29.432893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33ab12-da8c-4cd7-80da-29c98cf523bd None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:29.433621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33ab12-da8c-4cd7-80da-29c98cf523bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:29.436072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33ab12-da8c-4cd7-80da-29c98cf523bd None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:29.468713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 599/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Sat Sep 17 20:54:29 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:54:31.879866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b5362-122f-4ce7-9ac9-dc4c0f96a3a1 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.880410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b5362-122f-4ce7-9ac9-dc4c0f96a3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.880410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b5362-122f-4ce7-9ac9-dc4c0f96a3a1 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.882615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 607/1206] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:31.888117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:31.888542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:31.888789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:31.912579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:31.913820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:31.990461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 600/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:54:32.287363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b5362-122f-4ce7-9ac9-dc4c0f96a3a1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:32.408752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b5362-122f-4ce7-9ac9-dc4c0f96a3a1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:32.447180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:32.447802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:32.448080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:32.448383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:32.459734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:54:32.464449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:32.468899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:32.469834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 608/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:54:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:32.471836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:32.515335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:32.586394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76128396-7dab-47fb-9faa-a7c02db5fa4c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:32.625371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:32.626096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:32.626646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:32.626988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:32.627758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:32.628123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:32.629595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:32.638590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 601/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:54:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:32.644119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:32.702153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:32.759325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72a22f-0be9-4c75-8ce8-5ba3b656d707 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:32.797536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:32.798202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:32.798483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:32.798754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:32.810907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:32.812923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:32.828966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:32.832886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:32.834027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:32.834027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:32.834027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:32.834027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 609/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:54:32.836116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:32.944705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:32.948349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dd77e-65f3-43e0-b450-b0cee72e1dda None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:32.987828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:32.988489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:32.988820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:32.989095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:32.989761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:32.990129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:32.991751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:33.005408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 602/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:33.011030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:33.133254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d53bf-8a9a-4d19-8aae-ece6fa3436a0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:33.135589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:33.176232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:33.180723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:33.180723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:33.180723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:33.192279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:33.193955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:54:33.208634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:33.209932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:33.211109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:33.218545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 610/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:33.219869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:33.332354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59105777-8c77-4e4e-ad90-f4b5f383ca9e None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:33.369815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:33.372764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:33.372764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:33.372764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:33.372764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:33.372764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:33.374221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:33.400234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 603/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 17 20:54:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:33.401638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:33.531832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1846d1-4367-4ba4-a91f-da81a1f9589a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:33.573705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:33.574308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:33.575184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:33.575377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:33.607408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:33.649223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:33.650771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:33.765986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 611/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:54:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:54:33.771846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:33.791478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:33.887180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4171f-4d0d-4a6d-9750-d23752078ce7 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:33.926961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:33.927592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:33.927867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:33.928132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:33.939484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:33.940967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:33.956704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:33.958324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:33.959399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:33.959399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:33.959399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:33.960836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 604/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:33.966234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:34.036569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:34.084419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fca46e-0460-486b-95b6-d4934b7011b2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:34.132347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:34.133141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:34.133420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:34.133832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:34.134584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:34.134918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:34.136603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:34.151832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 612/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 20:54:34.158740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:34.243508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:34.281010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709e984-050f-43b7-8902-1dac8a2a1a49 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:34.322127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:34.322875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:34.323142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:34.323472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:34.331643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:34.333154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:54:34.347332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:34.348533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:34.349493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 605/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:54:34.356873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:34.492203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5f967-4b85-4808-9312-27c3694ca16e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:34.517399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:34.530570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:34.531188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:34.531533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:34.531861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:34.532635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:34.533029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:34.534466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67268ca-1735-4132-8ac9-41269ae00c83 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:34.547504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 613/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:34.554102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:34.664543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf7346-9c9c-4e87-9b1d-bb95f6b24458 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:34.694967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:34.701903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:34.702939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:34.703270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:34.703684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:34.704553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=8df169f9be474cf6b773aaf4addf0020) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:34.704954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:34.706449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc796c33-ac64-4751-a0cf-32fe27f72114 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:34.712745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 606/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:54:34 2022] GET /identity/v3/role_assignments?user.id=8df169f9be474cf6b773aaf4addf0020&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:36.953605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987bf32-144d-40b9-809d-fb65c894e70d None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:36.954131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987bf32-144d-40b9-809d-fb65c894e70d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:36.954244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987bf32-144d-40b9-809d-fb65c894e70d None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:36.955618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 614/1220] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:54:36.961220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:36.961626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:36.961765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:36.982263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:36.983450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:37.056346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 607/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:54:37.326314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987bf32-144d-40b9-809d-fb65c894e70d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:37.437766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987bf32-144d-40b9-809d-fb65c894e70d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:37.475563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02b215-2337-415c-989c-7007067258cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:37.476179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02b215-2337-415c-989c-7007067258cb None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:37.476456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02b215-2337-415c-989c-7007067258cb None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:37.476760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02b215-2337-415c-989c-7007067258cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:37.487919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02b215-2337-415c-989c-7007067258cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-8e02b215-2337-415c-989c-7007067258cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:54:37.492632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:37.494183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:37.495231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 615/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:54:37.499806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:37.608488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4461c-c07b-47d6-a15d-f9f875296218 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:37.633747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:37.646203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:37.646869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:37.647088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:37.647364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:37.648042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:37.648319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:37.649709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:37.658291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 608/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:37.664313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02b215-2337-415c-989c-7007067258cb None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:37.709144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:37.771696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02b215-2337-415c-989c-7007067258cb None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:37.806383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:37.807002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:37.807274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:37.807546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:37.818154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:37.819614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:37.835070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:37.841053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:37.842164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:37.842164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:37.842164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:37.842164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 616/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:37.842436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:37.952102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f6bea-d311-411e-b85d-bb33463f11bf None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:37.991339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:37.992166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:37.992298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:37.992513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:37.993327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:37.993660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:37.994998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:38.012722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 609/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:38.015105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:38.116064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:38.124119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937089e-33f1-44de-93d0-e03b4bf594f2 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:38.167989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:38.169054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:38.169373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:38.169695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:38.181110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:38.182578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:54:38.194858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:38.196017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:38.197218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:38.201588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 617/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:38.203115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:38.252962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:38.314736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1246c5-131a-4599-990f-0d38c42a5b0b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:38.349433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:38.350033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:38.350306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:38.350578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:38.351302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:38.351621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:38.353112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:38.372569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 610/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 17 20:54:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:38.379154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:38.505091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e4d0-b146-4d12-b9ed-455a892e449b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:38.542831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:38.544090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:38.544439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:38.544736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:38.555951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:38.557408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:38.574145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:38.575724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:38.576913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:38.576913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:38.576913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:38.576913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 618/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:38.581654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:38.684624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d8aa3-35c9-442d-8941-e526e8b6c7d2 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:38.687416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:38.724869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:38.725524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:38.725846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:38.726157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:38.726926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:38.727192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:38.729170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:38.741500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 611/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:38.747932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:38.760721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:38.859663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a96aa-5091-42ac-9d88-c90f54a16e18 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:38.895117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:38.895767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:38.896046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:38.896321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:38.904496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:38.906004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:54:38.918711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:38.919863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:38.920837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 619/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:38.925934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:39.040389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832819f-2a23-4a91-9c84-5e142704d0de None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:39.080254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:39.080893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:39.081177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:39.081455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:39.082151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:39.082457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:39.083882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe387f-2947-4158-910d-fa0f7c0be77d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:39.098168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 612/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:39.104158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:39.190719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:39.229472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c7ec8-0b25-4f63-a178-b4898a5534a3 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:39.266684 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbc337-128b-4b4e-b1b6-eae75bb0f07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:39.267442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc337-128b-4b4e-b1b6-eae75bb0f07e None admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:39.267753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc337-128b-4b4e-b1b6-eae75bb0f07e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:39.268156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc337-128b-4b4e-b1b6-eae75bb0f07e None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/8df169f9be474cf6b773aaf4addf0020/roles/1f93bc3abf5041708f35a3b0976c7f19` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:39.284113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbc337-128b-4b4e-b1b6-eae75bb0f07e None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=8df169f9be474cf6b773aaf4addf0020, role_id=1f93bc3abf5041708f35a3b0976c7f19)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:39.285601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbc337-128b-4b4e-b1b6-eae75bb0f07e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:39.298251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 620/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:54:39 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/8df169f9be474cf6b773aaf4addf0020/roles/1f93bc3abf5041708f35a3b0976c7f19 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:41.532904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8228d-21bd-4397-996c-6e6110526bd3 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:41.533460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8228d-21bd-4397-996c-6e6110526bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:41.533559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8228d-21bd-4397-996c-6e6110526bd3 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:41.535191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 613/1233] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:41.540873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:41.541297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:41.541431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:41.561865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:41.563054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:41.643839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 621/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:54:41.951443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8228d-21bd-4397-996c-6e6110526bd3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:41.981245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:42.067589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8228d-21bd-4397-996c-6e6110526bd3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:42.104052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:42.105763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:42.106185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:42.106463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:42.119630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:54:42.125975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:42.127290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:42.128248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 614/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Sat Sep 17 20:54:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:42.134173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:42.195123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:42.258036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f78b8-e602-43af-af76-adb839702a89 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:42.301360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:42.301963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:42.302241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:42.302514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:42.303176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:42.303476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:42.304896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:42.315453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 622/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Sat Sep 17 20:54:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:42.321423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:42.412466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:42.430517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125988c8-9890-4176-b8e8-3e4e16b91a42 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:42.471217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:42.471764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:42.472062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:42.472320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:42.483308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:42.484788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:42.503794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:42.504897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:42.506013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:42.506013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:42.506013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:42.506013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 615/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:54:42.512124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:42.630755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e621b4-c1ff-41fb-8f53-dbca404e66a6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:42.646229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:42.690354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:42.691102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:42.691481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:42.691766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:42.692724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:42.693050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:42.694719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:42.709110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 623/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:42.716461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:42.837096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596cd47-ac57-4119-9d5d-fe8825bf68ea None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:42.876570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:42.877241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:42.877517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:42.877787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:42.880880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:42.887646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:42.889510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:54:42.904197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:42.905392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:42.906646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:42.912488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 616/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:42.912634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.023928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83277-bd22-4426-9115-04705429a91a None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.062588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.063257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.063502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.063773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.064501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:43.064828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.066263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.084900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 624/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 17 20:54:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:43.091601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.205126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f73b67-c61d-4094-b4bd-617e5abb7fed None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.245317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.246824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.247261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.247667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.316721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:43.323787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.326121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.438502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 617/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:54:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:54:43.439718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.469932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:43.550625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70992581-4e09-4f22-8d95-270f95ca7ba6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.593995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.594660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.594975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.595240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.607127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.608665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:43.625283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:43.626572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:43.627710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:43.627710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:43.627710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:43.627710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 625/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 17 20:54:43.632745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.733082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:43.743226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72f06-f4f8-43e7-8d44-cfb6f3804ff2 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.782412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.783124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.783481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.783818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.784531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:43.784992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.786711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.800301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 618/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:43.806048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:43.893997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:43.918540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af2edf-bb4c-4b5d-9276-2b898b950d72 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:43.958069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:43.958694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:43.958977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:43.959249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:43.968451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:43.970542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:54:43.986373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:43.987470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:43.988505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 626/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:54:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:43.993683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:44.108467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57363c10-63d4-44ca-9c75-652ed096c167 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:44.145399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:44.145999 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:44.146272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:44.146540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:44.147257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:44.147533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:44.148965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69113755-59bb-4207-ae1a-54cf3700d7c7 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:44.162879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 619/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:44.170660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:44.281043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9ef28-f2a3-4f90-85c9-7e379b08028e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:44.317736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:44.318606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:44.318882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:44.319150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:44.319875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] RBAC: Adding query filter params (role.id=1f93bc3abf5041708f35a3b0976c7f19, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=8df169f9be474cf6b773aaf4addf0020) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:44.320220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:44.321982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf9816-790a-4212-970d-fa2835927e9e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:44.329770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 627/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:54:44 2022] GET /identity/v3/role_assignments?user.id=8df169f9be474cf6b773aaf4addf0020&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=1f93bc3abf5041708f35a3b0976c7f19 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:46.574682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68da84-38c8-443f-ae21-8bb01fff9436 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:46.575247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68da84-38c8-443f-ae21-8bb01fff9436 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:46.575247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68da84-38c8-443f-ae21-8bb01fff9436 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:46.576546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 620/1247] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:46.582529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:46.582996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:46.583151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:46.590807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:46.604042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:46.606033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:46.679685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 628/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:54:46.947197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68da84-38c8-443f-ae21-8bb01fff9436 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:46.966585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:47.053322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68da84-38c8-443f-ae21-8bb01fff9436 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:47.088111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:47.088778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:47.089059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:47.089315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:47.103336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:54:47.109137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:47.110270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:47.111114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 621/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:54:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:47.116529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:47.229437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcc39e-5d20-47e2-9a5d-a958a0fca85c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:47.259879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:47.265755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:47.266351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:47.266624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:47.266892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:47.267553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:47.267848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:47.269247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:47.278219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 629/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:47.284228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:47.360758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:47.385214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553446d2-557d-4af6-882b-ef934790d17f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:47.421927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:47.422640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:47.422978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:47.423295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:47.434372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:47.436108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:47.452966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:47.454000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:47.454933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:47.454933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:47.454933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:47.454933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 622/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:47.460386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:47.565411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0509f67-fa31-471b-b4c2-49b2d36513a0 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:47.601068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:47.601800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:47.601935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:47.602243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:47.602839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:47.603135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:47.604503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:47.616802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 630/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:47.623433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:47.730505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5506a-c6ac-404e-bdfc-0a431478efac None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:47.758347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:47.767529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:47.768173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:47.768447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:47.768740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:47.777252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:47.778713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:54:47.790970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:47.792157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:47.793339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 623/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:47.798882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:47.857560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:47.905664 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83438d26-f8f8-4b30-b38a-457ccb25d47d None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:47.941676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:47.942282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:47.942553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:47.942823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:47.943563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:47.943863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:47.945353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:47.965663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 631/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 17 20:54:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:47.971668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:48.075105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9151c07d-e75f-465d-8785-4819e3eb066b None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:48.109608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:48.110135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:48.110408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:48.110673 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:48.175837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:48.191159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:48.192610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:48.294861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 624/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:54:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:54:48.300733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:48.406108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb68ba53-5b15-4cd1-bd33-12301b596868 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:48.419884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:48.440145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:48.440793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:48.441069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:48.441341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:48.450847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:48.452279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:48.466923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:48.467988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:48.469037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:48.469037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:48.469037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:48.469037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 632/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:48.473697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:48.557148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:48.568904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29b9b-de89-44ce-8bb0-c52dd846807f None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:48.601389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:48.601974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:48.602247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:48.602514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:48.603138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:48.603430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:48.604790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:48.615951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 625/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:48.621501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:48.718459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e413a-1422-426d-a8f7-6da38861ebf6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:48.750858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:48.751459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:48.751736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:48.752007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:48.758380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:48.759815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:54:48.771194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:48.772223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:48.773511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 633/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:48.778663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:48.873958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4eb908-5e8f-4565-ad5c-a59bf293a282 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:48.906427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:48.907042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:48.907321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:48.907593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:48.908274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:48.908578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:48.910026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b682d14-212a-42d9-b81c-b6a57c2e713b None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:48.921580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 626/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:48.927505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:48.952811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:49.023356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f02a-7f36-46c3-86f1-64418bc2ed17 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:49.055468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:49.056047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:49.056323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:49.056594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:49.057336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=8df169f9be474cf6b773aaf4addf0020) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:49.057637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:49.059083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a481905-b894-40e1-a417-138cad0b8c47 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:49.064209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 634/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:54:48 2022] GET /identity/v3/role_assignments?user.id=8df169f9be474cf6b773aaf4addf0020&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=8a4cd71b629943809347939c4c950275 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:51.154253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ce181-93fa-4fbf-a4b3-607f2f217b1f None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:51.154784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ce181-93fa-4fbf-a4b3-607f2f217b1f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:51.154784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ce181-93fa-4fbf-a4b3-607f2f217b1f None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:51.156396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 627/1261] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:51.162711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:51.163402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:51.163402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:51.182465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:51.184411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:51.249311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 635/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 17 20:54:51.520552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ce181-93fa-4fbf-a4b3-607f2f217b1f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:51.532326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:51.622767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ce181-93fa-4fbf-a4b3-607f2f217b1f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:51.655771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:51.656385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:51.656663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:51.656962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:51.666624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:54:51.672321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:51.673412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:51.674313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 628/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:51.679413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:51.697002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:51.775359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36cae9-e634-416b-8cc7-f3efccdecd77 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:51.808497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:51.809112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:51.809386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:51.809654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:51.810327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:51.810620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:51.811986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:51.819607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 636/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:51.826032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:51.890432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:51.922391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fb7180-c41c-4d97-aa22-42d0f311ab4c None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:51.954924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:51.955526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:51.955803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:51.956073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:51.966201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:51.967630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:51.982481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:51.983562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:51.984457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:51.984457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:51.984457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:51.984457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 629/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:51.989247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.068021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:52.087499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c43120-6d77-4c7f-b39d-84fa36ae6fce None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:52.120140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:52.121295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:52.121572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:52.121843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:52.122478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:52.122770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:52.124141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:52.137006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 637/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:52.143621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.240490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187f1c-4257-44f8-9d06-80083f0df4a8 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:52.259352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:52.273669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:52.274281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:52.274558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:52.274828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:52.281934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:52.283356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:54:52.294258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:52.295323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:52.296527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 630/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:52.301338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.399353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67769545-2b22-4a23-8d3d-903de7f31f41 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:52.431701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:52.432298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:52.432579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:52.432885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:52.434252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:52.434556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:52.435979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:52.444241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:52.453662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 638/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 17 20:54:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:52.459529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.553927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf350a1-c0b9-4036-a8b4-9ec415520162 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:52.585838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:52.586437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:52.586711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:52.586978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:52.596988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:52.598402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:52.612287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:52.613407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:52.614484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:52.614484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:52.614484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:52.614484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 631/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:52.619065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.714170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568659e7-9fa8-4749-ba6b-f57a2550da42 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:52.747455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:52.748043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:52.748315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:52.748607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:52.749261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:52.749554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:52.750904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:52.761809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 639/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:52.767364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.799542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:52.867405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9145971-5c74-4dd2-84d9-094c61921548 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:52.899010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:52.899624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:52.899905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:52.900185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:52.907086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:52.908518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:54:52.920058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:52.921189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:52.922225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 632/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:52.927087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:52.985569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:53.025565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f93501-c3cc-46fb-bf33-0a3f397bd154 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:53.058777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:53.059365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:53.059641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:53.059910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:53.060589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:53.060917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:53.062332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9854756f-f7e2-4916-af28-10b731d5fb8d None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:53.073864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 640/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:53.080060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:53.177029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:53.185576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928c533-0b7b-48f5-b5c7-72c71129e811 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:53.217954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b9246-548a-4038-9103-2801b374a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:53.218598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9246-548a-4038-9103-2801b374a928 None admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:53.218880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9246-548a-4038-9103-2801b374a928 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:53.219149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9246-548a-4038-9103-2801b374a928 None admin] PATH_INFO: `/v3/projects/ed7f4752773441afbc91d004aa291020/users/8df169f9be474cf6b773aaf4addf0020/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:53.221014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b9246-548a-4038-9103-2801b374a928 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7f4752773441afbc91d004aa291020, user_id=8df169f9be474cf6b773aaf4addf0020, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:53.222424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b9246-548a-4038-9103-2801b374a928 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:53.234811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 633/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Sat Sep 17 20:54:53 2022] PUT /identity/v3/projects/ed7f4752773441afbc91d004aa291020/users/8df169f9be474cf6b773aaf4addf0020/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 20:54:55.344514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c2f9e-c213-4038-97c0-12e15f131bb6 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:55.345075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c2f9e-c213-4038-97c0-12e15f131bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:55.345075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c2f9e-c213-4038-97c0-12e15f131bb6 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:55.346435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 641/1274] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:55.351587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04455242-10d3-4243-a93c-74b40b915241 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:55.352001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04455242-10d3-4243-a93c-74b40b915241 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:55.352130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04455242-10d3-4243-a93c-74b40b915241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:55.371500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-04455242-10d3-4243-a93c-74b40b915241 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:55.372690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-04455242-10d3-4243-a93c-74b40b915241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:55.437228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 634/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:54:55.713777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c2f9e-c213-4038-97c0-12e15f131bb6 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:55.812868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c2f9e-c213-4038-97c0-12e15f131bb6 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:55.829783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:55.846794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:55.847403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:55.847677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:55.847949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:55.858299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 17 20:54:55.862949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:55.864004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:55.865058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 642/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Sat Sep 17 20:54:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:55.870001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04455242-10d3-4243-a93c-74b40b915241 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:55.947471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:55.966203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04455242-10d3-4243-a93c-74b40b915241 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:55.998761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:55.999351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:55.999626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:55.999895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:56.000547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:56.000872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:56.002229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:56.009887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 635/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Sat Sep 17 20:54:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:56.015937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:56.112696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029ae77-01de-43e3-b47f-0995e4fe4fc5 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:56.147546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:56.148698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:56.149122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:56.149547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:56.163266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:56.164693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:56.180340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:56.182072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:56.183544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:56.183544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:56.183544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:56.183544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 643/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:56.187750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:56.290985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec30d78-b794-499b-8543-e14f980106a9 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:56.322144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:56.322714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:56.322988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:56.323253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:56.324492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:56.324808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:56.326157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:56.337439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 636/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:56.344000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:56.373998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:56.442997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa31aa-e830-4379-a3f0-589b6ed032f9 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:56.476070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:56.476706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:56.476988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:56.477259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] PATH_INFO: `/v3/users/placement` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:56.484459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:56.485936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 17 20:54:56.496696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:56.498604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:54:56.499741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 644/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:56.503823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:56.507655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:56.600586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e03712-7603-44a1-ae38-4d8ca1641f7a None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:56.633141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:56.633726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:56.633998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:56.634263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:56.634976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:56.635272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:56.637327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:56.653871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 637/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Sat Sep 17 20:54:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:56.659602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:56.746957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:56.756389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecf512-0bd7-4ff5-a27a-8c2d2cf9faca None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:56.788633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:56.789242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:56.789518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:56.789792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:56.864571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:56.865956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:56.956065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:56.966365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 645/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Sat Sep 17 20:54:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 17 20:54:56.972203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:57.030341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:57.066947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fd00d-6d4f-4fca-8069-937076f85f42 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:57.098713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:57.099313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:57.099586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:57.099856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:57.109958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:57.111376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 17 20:54:57.126706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:57.127693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 17 20:54:57.128657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 17 20:54:57.128657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 17 20:54:57.128657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:57.128657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 638/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:57.133787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:57.243837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa302624-990b-40c9-94fe-f815e9048d30 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:57.276854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:57.277626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:57.277905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:57.278174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:57.278804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:57.279101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:57.280455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:57.291467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 646/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Sat Sep 17 20:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:57.298431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:57.399495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e9ad4-7054-4d2f-b682-126648ca98c6 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:57.413208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:57.433915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:57.434539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:57.434818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:57.435091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:57.441461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:57.442898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 17 20:54:57.454436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application with Lock( Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:54:57.455451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: ERROR keystone.server.flask.application Sep 17 20:54:57.456737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 639/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Sat Sep 17 20:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:54:57.461445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:57.511279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:57.559810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a28d1-1c9e-4d3d-9757-1d797e99373b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:57.592003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:57.592594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:57.592893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:57.593162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:57.593854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:57.594151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:57.595577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b0d39-c619-4b3d-8164-7a1e333fa13e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:57.607705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 647/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Sat Sep 17 20:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:57.613770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:54:57.709343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb23bbc-432a-47f6-bed7-f0bdec95c458 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:54:57.742121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:54:57.742703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:57.742979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:57.743249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:57.743963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] RBAC: Adding query filter params (role.id=8a4cd71b629943809347939c4c950275, scope.project.id=ed7f4752773441afbc91d004aa291020, user.id=8df169f9be474cf6b773aaf4addf0020) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:54:57.744266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:54:57.745756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc409de-3822-4c2b-8688-58b48b07cc75 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:54:57.751250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 640/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Sat Sep 17 20:54:57 2022] GET /identity/v3/role_assignments?user.id=8df169f9be474cf6b773aaf4addf0020&scope.project.id=ed7f4752773441afbc91d004aa291020&role.id=8a4cd71b629943809347939c4c950275 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:54:59.939696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe8e15-82d8-46fa-936a-7723420e55d6 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:59.940248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe8e15-82d8-46fa-936a-7723420e55d6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:59.940248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe8e15-82d8-46fa-936a-7723420e55d6 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:59.941602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 648/1288] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:54:59.946791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:54:59.947159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:54:59.947332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:54:59.966091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:54:59.967252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:54:59.996044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:00.030719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 641/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:54:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:55:00.294783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe8e15-82d8-46fa-936a-7723420e55d6 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:00.394291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe8e15-82d8-46fa-936a-7723420e55d6 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:00.405732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:00.429100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:00.429797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:00.430073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:00.430341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:00.430912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:00.432268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:55:00.440802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:55:00.441949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:00.442982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 649/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:55:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:55:00.447854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:00.542572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114585db-c219-4ba2-aea0-4e746c9b8777 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:00.574421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:00.574939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:00.575214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:00.575482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:00.576093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:00.576386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:00.578028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9617b7b-8d7d-42c5-acac-1587cb3c7dc2 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:00.582601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 642/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 17 20:55:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:00.588262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:00.687055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350a95f-317e-4b95-abac-5ad1ad632a4e None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:00.719331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:00.719916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:00.720188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:00.720454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] PATH_INFO: `/v3/services` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:00.721097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:00.721393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:00.722795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39213e6e-0b9a-47d8-90de-bf448a8147c1 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:00.727004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 650/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 17 20:55:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:02.836318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06470b-6637-4d39-ac78-4c4fbe46d4e1 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:02.836867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06470b-6637-4d39-ac78-4c4fbe46d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:02.836867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06470b-6637-4d39-ac78-4c4fbe46d4e1 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:02.838224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 643/1293] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:55:02.843667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692a19f-f778-4481-9b64-d2a0aeb4370a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:02.844097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692a19f-f778-4481-9b64-d2a0aeb4370a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:02.844197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692a19f-f778-4481-9b64-d2a0aeb4370a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:02.863286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5692a19f-f778-4481-9b64-d2a0aeb4370a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:02.865395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5692a19f-f778-4481-9b64-d2a0aeb4370a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:02.896572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:02.933690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 651/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:55:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:55:03.209866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06470b-6637-4d39-ac78-4c4fbe46d4e1 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:03.250251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:03.317003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06470b-6637-4d39-ac78-4c4fbe46d4e1 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:03.349255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ffb3f-e51a-42f7-8161-f1e9471cdd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:03.349839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ffb3f-e51a-42f7-8161-f1e9471cdd0d None admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:03.350112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ffb3f-e51a-42f7-8161-f1e9471cdd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:03.350403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ffb3f-e51a-42f7-8161-f1e9471cdd0d None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:03.351037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ffb3f-e51a-42f7-8161-f1e9471cdd0d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:03.352372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ffb3f-e51a-42f7-8161-f1e9471cdd0d None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:03.366558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 644/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Sat Sep 17 20:55:03 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:05.488112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8d40-3270-441c-be4d-ba7b61c837ec None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:05.488653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8d40-3270-441c-be4d-ba7b61c837ec None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:05.488653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8d40-3270-441c-be4d-ba7b61c837ec None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:05.490027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 652/1296] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:55:05.495266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:05.495640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:05.495839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:05.514581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:05.515742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:05.584960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 645/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:55:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:55:05.854836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de8d40-3270-441c-be4d-ba7b61c837ec None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:05.954219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de8d40-3270-441c-be4d-ba7b61c837ec None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:05.986770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:05.987380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:05.987658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:05.987932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:05.988509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:05.989898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:55:05.999127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:55:06.000275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:06.001376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 653/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:55:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:55:06.006292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:06.023887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:06.101058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcbbb5-2a41-47ca-a20f-b30ffc07b1a4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:06.134740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:06.135264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:06.135606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:06.135876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:06.136505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:06.136963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:06.138196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:06.143301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 646/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 17 20:55:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:06.149696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:06.186470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:06.255606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ceec78-507d-464f-be95-645afe7939f6 None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:06.287248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:06.287831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:06.288104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:06.288370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:06.289094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] RBAC: Adding query filter params (interface=public, service_id=d80afc88382d446c99362daf3a74b034, region_id=RegionOne) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:06.289393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:06.290746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a932c72-9004-444f-bd12-500d584e1569 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:06.296163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 654/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Sat Sep 17 20:55:06 2022] GET /identity/v3/endpoints?service_id=d80afc88382d446c99362daf3a74b034&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:06.301869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:06.380325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:06.397773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26593eb-166e-4e04-a061-ac3cd0541699 None admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:06.431067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29052423-a609-42d7-ada9-13f4dc24d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:06.431657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29052423-a609-42d7-ada9-13f4dc24d307 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:06.431934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29052423-a609-42d7-ada9-13f4dc24d307 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:06.432202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29052423-a609-42d7-ada9-13f4dc24d307 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:06.432854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29052423-a609-42d7-ada9-13f4dc24d307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:06.434204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29052423-a609-42d7-ada9-13f4dc24d307 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:06.440809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 647/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Sat Sep 17 20:55:06 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:55:08.540052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88180088-b925-4b9a-86a5-d072d32cf023 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:08.540566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88180088-b925-4b9a-86a5-d072d32cf023 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:08.540566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88180088-b925-4b9a-86a5-d072d32cf023 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:08.542104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 655/1302] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:55:08.547165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:08.547542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:08.547711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:08.565343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:55:08.566498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:55:08.630488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 648/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:55:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 17 20:55:08.915022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88180088-b925-4b9a-86a5-d072d32cf023 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:09.007760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:09.014917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88180088-b925-4b9a-86a5-d072d32cf023 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:09.047337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:09.047945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:09.048218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:09.048482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:09.049084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:09.050434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 17 20:55:09.059399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 17 20:55:09.060525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.get_or_create( Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application with Lock( Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self._enter() Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application return self.creator() Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application created_value = creator( Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: ERROR keystone.server.flask.application Sep 17 20:55:09.061417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 656/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Sat Sep 17 20:55:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 17 20:55:09.066373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:09.138844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:09.160868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b38cf-383e-443e-93ba-b2762f9b5fac None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:09.194022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:09.194609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:09.194888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:09.195213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] PATH_INFO: `/v3/services` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:09.195838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 20:55:09.196137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:09.197529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ec33-afb7-4f75-8afd-991459e73a48 None admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:09.203168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 649/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Sat Sep 17 20:55:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 20:55:09.208847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:55:09.305558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5884b0-75ab-472b-9693-dccfa4ed287b None admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:55:09.338874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8f563-3f32-4653-b62a-c9aa82c3e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:55:09.339457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8f563-3f32-4653-b62a-c9aa82c3e5f0 None admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:55:09.339732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8f563-3f32-4653-b62a-c9aa82c3e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:55:09.340530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8f563-3f32-4653-b62a-c9aa82c3e5f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:55:09.341209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8f563-3f32-4653-b62a-c9aa82c3e5f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:55:09.342554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8f563-3f32-4653-b62a-c9aa82c3e5f0 None admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:55:09.361517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 657/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 20:55:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 20:55:19.199487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=30 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:19.357443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:13.328294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e7b43-8734-42f5-99ea-4e7306f55253 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:13.328789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e7b43-8734-42f5-99ea-4e7306f55253 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:13.329026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e7b43-8734-42f5-99ea-4e7306f55253 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:13.330560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 650/1307] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:13.339793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cc0f1-6409-4c7b-a87a-5c97c0f0a356 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:13.341171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cc0f1-6409-4c7b-a87a-5c97c0f0a356 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:13.342011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cc0f1-6409-4c7b-a87a-5c97c0f0a356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:13.474014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f74cc0f1-6409-4c7b-a87a-5c97c0f0a356 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:13.475359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f74cc0f1-6409-4c7b-a87a-5c97c0f0a356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:13.589022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 658/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:13.885508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f018bc1-1fcc-411f-b85e-1f8bdf1233b7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:13.885944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f018bc1-1fcc-411f-b85e-1f8bdf1233b7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:13.886488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f018bc1-1fcc-411f-b85e-1f8bdf1233b7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:13.888053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 651/1309] 10.222.0.16 () {54 vars in 963 bytes} [Sat Sep 17 20:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:13.894448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81775f5-989b-442e-bb7c-7690b02a9a1d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:13.894868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81775f5-989b-442e-bb7c-7690b02a9a1d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:13.895252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81775f5-989b-442e-bb7c-7690b02a9a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:13.974884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e81775f5-989b-442e-bb7c-7690b02a9a1d None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:13.976248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e81775f5-989b-442e-bb7c-7690b02a9a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:13.998218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:14.058143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 659/1310] 10.222.0.16 () {58 vars in 1060 bytes} [Sat Sep 17 20:56:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:14.067385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f018bc1-1fcc-411f-b85e-1f8bdf1233b7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:14.069140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6f018bc1-1fcc-411f-b85e-1f8bdf1233b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:14.194385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f018bc1-1fcc-411f-b85e-1f8bdf1233b7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:14.278405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:14.279050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:14.279330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:14.279602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:14.281238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:14.339938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:14.405788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:14.407226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c69de-d42b-4fc7-827d-2e87638acfbf service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:14.426413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 652/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:16.891120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bb34e-8e83-4796-8aa8-f959ab9f3273 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:16.891637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bb34e-8e83-4796-8aa8-f959ab9f3273 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:16.891637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bb34e-8e83-4796-8aa8-f959ab9f3273 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:16.892961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 660/1312] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:16.898441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1841-fd91-4fc9-a456-fa1c43632120 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:16.898845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1841-fd91-4fc9-a456-fa1c43632120 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:16.898987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1841-fd91-4fc9-a456-fa1c43632120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:16.980981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2dfc1841-fd91-4fc9-a456-fa1c43632120 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:16.982188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2dfc1841-fd91-4fc9-a456-fa1c43632120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:17.095546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 653/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:17.379541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bb34e-8e83-4796-8aa8-f959ab9f3273 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:17.381085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-cc4bb34e-8e83-4796-8aa8-f959ab9f3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:17.493215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bb34e-8e83-4796-8aa8-f959ab9f3273 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:17.572429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:17.574819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:17.575508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:17.575857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:17.576637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:17.578790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:17.690988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:17.692438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296460c-16b8-4dda-a4f6-fa1cef9adf35 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:17.710931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 661/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:20.169015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b9cb-5294-4aec-858c-198b583f5720 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:20.169581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b9cb-5294-4aec-858c-198b583f5720 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:20.169581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b9cb-5294-4aec-858c-198b583f5720 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:20.170895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 654/1315] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:20.176177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841b6f-60e3-4d86-9338-7925cf09bca1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:20.176603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841b6f-60e3-4d86-9338-7925cf09bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:20.176758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41841b6f-60e3-4d86-9338-7925cf09bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:20.241096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-41841b6f-60e3-4d86-9338-7925cf09bca1 None None] MFA Rules not processed for user `e925dc6d06704d19b20429d1596b7d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:20.242294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-41841b6f-60e3-4d86-9338-7925cf09bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:20.294067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:20.338436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 662/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:20.643413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d2ce-acaa-47bb-ac41-4e1cdbde3df7 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:20.643694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d2ce-acaa-47bb-ac41-4e1cdbde3df7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:20.643966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17d2ce-acaa-47bb-ac41-4e1cdbde3df7 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:20.645377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 655/1317] 10.222.0.16 () {54 vars in 963 bytes} [Sat Sep 17 20:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:20.651594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2edde-5cf8-4737-89eb-aac7cecf49e2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:20.651873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2edde-5cf8-4737-89eb-aac7cecf49e2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:20.652144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2edde-5cf8-4737-89eb-aac7cecf49e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:20.725860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-84d2edde-5cf8-4737-89eb-aac7cecf49e2 None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:20.727054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-84d2edde-5cf8-4737-89eb-aac7cecf49e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:20.799543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 663/1318] 10.222.0.16 () {58 vars in 1060 bytes} [Sat Sep 17 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:20.809052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17d2ce-acaa-47bb-ac41-4e1cdbde3df7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:20.810599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6e17d2ce-acaa-47bb-ac41-4e1cdbde3df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:20.820054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:20.911954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17d2ce-acaa-47bb-ac41-4e1cdbde3df7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:20.984338 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:20.984969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:20.985248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:20.985520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:20.987112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:21.047423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:21.094030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:21.095401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca4c6d-9116-46be-86dc-a1c556ce3ecc service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:21.113311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 656/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:24.153651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1612be-60bf-4516-aeb8-9ac993fc362c None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:24.154183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1612be-60bf-4516-aeb8-9ac993fc362c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:24.154183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1612be-60bf-4516-aeb8-9ac993fc362c None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:24.155549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 664/1320] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:24.160896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83698b6b-e10c-469f-9386-81581226612f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:24.161285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83698b6b-e10c-469f-9386-81581226612f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:24.161444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83698b6b-e10c-469f-9386-81581226612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:24.225730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-83698b6b-e10c-469f-9386-81581226612f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:24.226924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-83698b6b-e10c-469f-9386-81581226612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:24.320943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 657/1321] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:24.605103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1612be-60bf-4516-aeb8-9ac993fc362c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:24.606633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9d1612be-60bf-4516-aeb8-9ac993fc362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:24.657654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:24.709937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1612be-60bf-4516-aeb8-9ac993fc362c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:24.780427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:24.780973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:24.781245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:24.781512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:24.783068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:24.874112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:24.887510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:24.888967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453018c1-6f2a-4b85-934f-abf12f139246 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:24.906924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 665/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:27.737900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66adac8-84c5-432b-af15-bf40a696ea90 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:27.738490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66adac8-84c5-432b-af15-bf40a696ea90 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:27.738490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66adac8-84c5-432b-af15-bf40a696ea90 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:27.739792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 658/1323] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:27.745946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0b89-f220-4e59-9800-c33bf17e8628 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:27.746388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0b89-f220-4e59-9800-c33bf17e8628 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:27.746667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b0b89-f220-4e59-9800-c33bf17e8628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:27.839064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f97b0b89-f220-4e59-9800-c33bf17e8628 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:27.840290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f97b0b89-f220-4e59-9800-c33bf17e8628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:27.946271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 666/1324] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:28.228748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66adac8-84c5-432b-af15-bf40a696ea90 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:28.230373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a66adac8-84c5-432b-af15-bf40a696ea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:28.250959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:28.337778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66adac8-84c5-432b-af15-bf40a696ea90 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:28.418808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:28.419415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:28.419697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:28.419971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:28.421614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:28.492911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:28.532727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:28.534215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef3b78-8724-4610-aee2-711503f637ca service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:28.551918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 659/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:30.766095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6421677-9517-4b58-83af-e94bf55cb7ce None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:30.766634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6421677-9517-4b58-83af-e94bf55cb7ce None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:30.766634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6421677-9517-4b58-83af-e94bf55cb7ce None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:30.767992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 667/1326] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:30.773233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519a17c-c5d9-4db9-af5a-3a448e951d38 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:30.773644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519a17c-c5d9-4db9-af5a-3a448e951d38 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:30.773787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519a17c-c5d9-4db9-af5a-3a448e951d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:30.857132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8519a17c-c5d9-4db9-af5a-3a448e951d38 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:30.858146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8519a17c-c5d9-4db9-af5a-3a448e951d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:30.953772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 660/1327] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:31.234536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6421677-9517-4b58-83af-e94bf55cb7ce None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:31.236064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f6421677-9517-4b58-83af-e94bf55cb7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:31.296308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:31.340926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6421677-9517-4b58-83af-e94bf55cb7ce None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:31.417169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:31.417801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:31.418074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:31.418345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:31.419930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:31.529230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:31.531340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:31.533365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220771a3-5756-42bb-9344-915a5de712f9 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:31.552315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 668/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:33.768283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c2adc-a946-4c2f-8dbd-a32193a70d7f None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:33.768823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c2adc-a946-4c2f-8dbd-a32193a70d7f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:33.768823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c2adc-a946-4c2f-8dbd-a32193a70d7f None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:33.770145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 661/1329] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:33.776288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209d022-01ee-4357-9bb9-aa3ef31942cd None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:33.776748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209d022-01ee-4357-9bb9-aa3ef31942cd None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:33.776863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209d022-01ee-4357-9bb9-aa3ef31942cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:33.845009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0209d022-01ee-4357-9bb9-aa3ef31942cd None None] MFA Rules not processed for user `e925dc6d06704d19b20429d1596b7d2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:33.846216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0209d022-01ee-4357-9bb9-aa3ef31942cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:33.940243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 669/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:34.236868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c2adc-a946-4c2f-8dbd-a32193a70d7f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:34.238682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a26c2adc-a946-4c2f-8dbd-a32193a70d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:34.271915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:34.345984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c2adc-a946-4c2f-8dbd-a32193a70d7f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:34.421179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:34.421783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:34.422057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:34.422325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:34.423913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:34.504845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:34.536040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:34.537509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c02a9-ceef-47bb-b210-f34a74cb5605 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:34.563492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 662/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:37.098527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be471-43d9-4775-99f0-9971490eac28 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:37.099029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be471-43d9-4775-99f0-9971490eac28 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:37.099029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be471-43d9-4775-99f0-9971490eac28 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:37.100375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 670/1332] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:37.106216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19d1d5-c8a1-48f8-b5bd-5164de4d330f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:37.106645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19d1d5-c8a1-48f8-b5bd-5164de4d330f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:37.106756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19d1d5-c8a1-48f8-b5bd-5164de4d330f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:37.172620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-cd19d1d5-c8a1-48f8-b5bd-5164de4d330f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:37.173862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cd19d1d5-c8a1-48f8-b5bd-5164de4d330f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:37.267930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 663/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:37.548720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be471-43d9-4775-99f0-9971490eac28 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:37.550259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5b5be471-43d9-4775-99f0-9971490eac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:37.619523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:37.654251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be471-43d9-4775-99f0-9971490eac28 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:37.725491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:37.726083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:37.726356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:37.726634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:37.728196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:37.831524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:37.832972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb783ca-8109-483a-bd71-a66df4ee337a service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:37.843024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:37.851966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 671/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:40.036932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e18587-4f0c-4692-981f-960fb54c8d47 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:40.037488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e18587-4f0c-4692-981f-960fb54c8d47 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:40.037488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e18587-4f0c-4692-981f-960fb54c8d47 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:40.038829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 664/1335] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:40.044054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ee149-b331-442d-8f85-8bc10e9f8b23 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:40.044449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ee149-b331-442d-8f85-8bc10e9f8b23 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:40.044603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ee149-b331-442d-8f85-8bc10e9f8b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:40.109405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-712ee149-b331-442d-8f85-8bc10e9f8b23 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:40.110641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-712ee149-b331-442d-8f85-8bc10e9f8b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:40.219565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 672/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:40.506971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e18587-4f0c-4692-981f-960fb54c8d47 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:40.508536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b0e18587-4f0c-4692-981f-960fb54c8d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:40.548598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:40.610674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e18587-4f0c-4692-981f-960fb54c8d47 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:40.683502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:40.684097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:40.684376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:40.684646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:40.686265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:40.766670 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:40.792401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:40.793863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029736c0-f2a1-43ce-af76-75d065337a98 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:40.811602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 665/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:43.277558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fee60e-c535-420a-8a6e-e0a9bcd8df14 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:43.278092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fee60e-c535-420a-8a6e-e0a9bcd8df14 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:43.278092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fee60e-c535-420a-8a6e-e0a9bcd8df14 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:43.279469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 673/1338] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:43.284765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41cebe-9ed8-4a1e-9479-dfd07d798a44 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:43.285213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41cebe-9ed8-4a1e-9479-dfd07d798a44 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:43.285344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41cebe-9ed8-4a1e-9479-dfd07d798a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:43.350285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-fd41cebe-9ed8-4a1e-9479-dfd07d798a44 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:43.351564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fd41cebe-9ed8-4a1e-9479-dfd07d798a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:43.449061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 666/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:43.732440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fee60e-c535-420a-8a6e-e0a9bcd8df14 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:43.733999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21fee60e-c535-420a-8a6e-e0a9bcd8df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:43.808436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:43.836871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fee60e-c535-420a-8a6e-e0a9bcd8df14 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:43.907859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:43.908452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:43.908748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:43.909023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:43.910597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:44.017125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:44.018485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727698d5-58f6-4d7b-9b00-4ee48c2ff941 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:44.035744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 674/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:48.117997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b33f9-1655-43b6-938b-59e0948142c6 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:48.118511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b33f9-1655-43b6-938b-59e0948142c6 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:48.118511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b33f9-1655-43b6-938b-59e0948142c6 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:48.120071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 667/1341] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:48.125258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69945c-14d5-4803-97bd-80ae645153d4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:48.125731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69945c-14d5-4803-97bd-80ae645153d4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:48.125841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69945c-14d5-4803-97bd-80ae645153d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:48.131132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:48.205505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7c69945c-14d5-4803-97bd-80ae645153d4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:48.206758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7c69945c-14d5-4803-97bd-80ae645153d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:48.317501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 675/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:48.603949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b33f9-1655-43b6-938b-59e0948142c6 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:48.605538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-869b33f9-1655-43b6-938b-59e0948142c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:48.654127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:48.710273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b33f9-1655-43b6-938b-59e0948142c6 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:48.784883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:48.785399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:48.785714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:48.785983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:48.788314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:48.881869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:48.899158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:48.900611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154d58a-ed8a-41ee-9293-a4602698cd03 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:48.919899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 668/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:56:51.732494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fb427-99a0-4398-80fc-72d5ce1b06de None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:51.733067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fb427-99a0-4398-80fc-72d5ce1b06de None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:51.733067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fb427-99a0-4398-80fc-72d5ce1b06de None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:51.734574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 676/1344] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:51.739664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ae16d-8b22-457c-a4c9-820127b503b3 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:51.740047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ae16d-8b22-457c-a4c9-820127b503b3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:51.740213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ae16d-8b22-457c-a4c9-820127b503b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:51.806514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b24ae16d-8b22-457c-a4c9-820127b503b3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:51.807710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b24ae16d-8b22-457c-a4c9-820127b503b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:51.904084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 669/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:52.185837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fb427-99a0-4398-80fc-72d5ce1b06de None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:52.187380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8d6fb427-99a0-4398-80fc-72d5ce1b06de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:52.277151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:52.296094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fb427-99a0-4398-80fc-72d5ce1b06de None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:52.373393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:52.373993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:52.374247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:52.374520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:52.376101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:52.485879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:52.487304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443a767-ff9a-4d79-8986-2c6e929814b6 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:52.506690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 677/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:56:57.055214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bfa92-9e34-4143-896c-42b596a981bb None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:57.055794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bfa92-9e34-4143-896c-42b596a981bb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:57.055794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bfa92-9e34-4143-896c-42b596a981bb None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:57.057171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 670/1347] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:56:57.062564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224e36-f153-4297-991e-98e4422ffcb7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:57.063004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224e36-f153-4297-991e-98e4422ffcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:57.063135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224e36-f153-4297-991e-98e4422ffcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:57.080770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:57.134815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-df224e36-f153-4297-991e-98e4422ffcb7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:56:57.136018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-df224e36-f153-4297-991e-98e4422ffcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:57.231495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 678/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:56:57.521065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bfa92-9e34-4143-896c-42b596a981bb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:56:57.522644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-733bfa92-9e34-4143-896c-42b596a981bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:57.579971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:57.624632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733bfa92-9e34-4143-896c-42b596a981bb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:56:57.696633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:56:57.697256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:56:57.697534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:56:57.697806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:56:57.699407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:56:57.802906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:57.807945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:56:57.809420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e1e425-bdaa-4650-a320-7927c50fde5a service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:56:57.827469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 671/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:02.800609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc8281-50f2-4cf6-8d6f-22dc0ffbc975 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:02.801121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc8281-50f2-4cf6-8d6f-22dc0ffbc975 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:02.801206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc8281-50f2-4cf6-8d6f-22dc0ffbc975 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:02.802869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 679/1350] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 20:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:02.809939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953ab14-af80-43ce-a531-d45d340f68b9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:02.809939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953ab14-af80-43ce-a531-d45d340f68b9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:02.809939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a953ab14-af80-43ce-a531-d45d340f68b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:02.948913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a953ab14-af80-43ce-a531-d45d340f68b9 None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:02.950396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a953ab14-af80-43ce-a531-d45d340f68b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:03.104499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 672/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 20:57:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:07.230015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=59 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:07.314319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084a85a-10af-4f61-a841-98ab7aec95fd None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:07.314678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084a85a-10af-4f61-a841-98ab7aec95fd None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:07.314908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084a85a-10af-4f61-a841-98ab7aec95fd None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:07.316381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 680/1352] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 20:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:57:07.321978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3d53-53b8-4892-8953-48be6b24c2b6 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:07.322476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3d53-53b8-4892-8953-48be6b24c2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:07.322760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3d53-53b8-4892-8953-48be6b24c2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:07.383389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:07.414811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c32b3d53-53b8-4892-8953-48be6b24c2b6 None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:07.416174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c32b3d53-53b8-4892-8953-48be6b24c2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:07.536713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 673/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 20:57:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:08.774437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e996d8-fbdf-4bf3-9e60-6b4fee79a3d5 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:08.774944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e996d8-fbdf-4bf3-9e60-6b4fee79a3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:08.775048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e996d8-fbdf-4bf3-9e60-6b4fee79a3d5 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:08.776370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 681/1354] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 20:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:08.781443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a665f-6198-40cc-88df-57610a6c1bc2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:08.781944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a665f-6198-40cc-88df-57610a6c1bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:08.782042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a665f-6198-40cc-88df-57610a6c1bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:08.859943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-121a665f-6198-40cc-88df-57610a6c1bc2 None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:08.861520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-121a665f-6198-40cc-88df-57610a6c1bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:08.904480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:08.969982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 674/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 20:57:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:57:12.599517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824ac2e-500f-479e-a4f8-1efb9040c2ab None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:12.600114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824ac2e-500f-479e-a4f8-1efb9040c2ab None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:12.600114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824ac2e-500f-479e-a4f8-1efb9040c2ab None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:12.601509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 682/1356] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:12.607068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c609-8200-4943-84b0-2686ff54443b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:12.607518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c609-8200-4943-84b0-2686ff54443b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:12.607613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c609-8200-4943-84b0-2686ff54443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:12.680945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b734c609-8200-4943-84b0-2686ff54443b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:12.682413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b734c609-8200-4943-84b0-2686ff54443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:12.784076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 675/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:13.102104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eed4c-bb95-44e3-8ddc-6e0b438c30ae None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:13.102806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eed4c-bb95-44e3-8ddc-6e0b438c30ae None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:13.103433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52eed4c-bb95-44e3-8ddc-6e0b438c30ae None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:13.105626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 683/1358] 10.222.0.16 () {54 vars in 965 bytes} [Sat Sep 17 20:57:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:13.111674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aca07f-6e57-4b79-be4c-57b25738231a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:13.111954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aca07f-6e57-4b79-be4c-57b25738231a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:13.112228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aca07f-6e57-4b79-be4c-57b25738231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:13.134258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:13.198678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a2aca07f-6e57-4b79-be4c-57b25738231a None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:13.199851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a2aca07f-6e57-4b79-be4c-57b25738231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:13.317202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 676/1359] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 17 20:57:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:13.324640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eed4c-bb95-44e3-8ddc-6e0b438c30ae None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:13.326440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c52eed4c-bb95-44e3-8ddc-6e0b438c30ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:13.486569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52eed4c-bb95-44e3-8ddc-6e0b438c30ae None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:13.572655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:13.617809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:13.618464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:13.618690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:13.618930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:13.620512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:13.718963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:13.720575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242159f0-0da6-4c2b-8b76-8519e1f19677 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:13.740165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 684/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:16.156802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d7811-cc67-4914-ab8e-0c8ad2a4da23 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:16.157339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d7811-cc67-4914-ab8e-0c8ad2a4da23 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:16.157339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d7811-cc67-4914-ab8e-0c8ad2a4da23 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:16.158702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 677/1361] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:16.164085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1faeb5-6f71-490a-b059-eacc8cd75bcf None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:16.164647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1faeb5-6f71-490a-b059-eacc8cd75bcf None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:16.164647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1faeb5-6f71-490a-b059-eacc8cd75bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:16.265758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ab1faeb5-6f71-490a-b059-eacc8cd75bcf None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:16.267185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ab1faeb5-6f71-490a-b059-eacc8cd75bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:16.306431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:16.373329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 685/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:16.665480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d7811-cc67-4914-ab8e-0c8ad2a4da23 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:16.667372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-426d7811-cc67-4914-ab8e-0c8ad2a4da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:16.741382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:16.837303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d7811-cc67-4914-ab8e-0c8ad2a4da23 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:16.975886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:16.976587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:16.976946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:16.977280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:16.980072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:17.012799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:17.084826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:17.086662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35161e85-6ce6-48f9-997e-1f6acdb1a4d5 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:17.105746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 678/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:19.610611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4ab61-3009-44a2-a7cb-23b3b4364602 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:19.611123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4ab61-3009-44a2-a7cb-23b3b4364602 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:19.611123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4ab61-3009-44a2-a7cb-23b3b4364602 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:19.612407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 686/1364] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:19.618805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37ebac-da0e-4ed0-abab-908d92bb1a13 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:19.619224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37ebac-da0e-4ed0-abab-908d92bb1a13 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:19.619410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37ebac-da0e-4ed0-abab-908d92bb1a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:19.691194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7d37ebac-da0e-4ed0-abab-908d92bb1a13 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:19.692420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7d37ebac-da0e-4ed0-abab-908d92bb1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:19.792021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:19.801145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 679/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:20.085989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fd82a-62f3-4785-9b68-626c96e43e38 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:20.086408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fd82a-62f3-4785-9b68-626c96e43e38 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:20.086784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fd82a-62f3-4785-9b68-626c96e43e38 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:20.088308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 687/1366] 10.222.0.16 () {54 vars in 965 bytes} [Sat Sep 17 20:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:20.093904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea0a23-ec71-47de-96d0-98112523af48 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:20.094188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea0a23-ec71-47de-96d0-98112523af48 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:20.094458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea0a23-ec71-47de-96d0-98112523af48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:20.175197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-42ea0a23-ec71-47de-96d0-98112523af48 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:20.176411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-42ea0a23-ec71-47de-96d0-98112523af48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:20.296050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 680/1367] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 17 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:20.303884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fd82a-62f3-4785-9b68-626c96e43e38 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:20.305534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-cf4fd82a-62f3-4785-9b68-626c96e43e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:20.469275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fd82a-62f3-4785-9b68-626c96e43e38 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:20.490682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:20.595952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:20.596331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:20.596606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:20.596917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:20.598832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:20.702171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:20.703819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59503c4b-578d-4857-bafe-226e7736b08d service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:20.722623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 688/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:20.911932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef626a5-7ca9-447c-b9bf-faaf7ec2cd67 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:20.912528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef626a5-7ca9-447c-b9bf-faaf7ec2cd67 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:20.912528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef626a5-7ca9-447c-b9bf-faaf7ec2cd67 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:20.914600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 681/1369] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:20.920394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b362f-46b3-49bd-aea2-a58267c1e537 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:20.920701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b362f-46b3-49bd-aea2-a58267c1e537 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:20.921009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b362f-46b3-49bd-aea2-a58267c1e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:20.959849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.005745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-998b362f-46b3-49bd-aea2-a58267c1e537 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:21.007560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-998b362f-46b3-49bd-aea2-a58267c1e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:21.116723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 689/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:21.416118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8522c7-ffab-40c9-8437-dad7ea15cae9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:21.416644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8522c7-ffab-40c9-8437-dad7ea15cae9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:21.417153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8522c7-ffab-40c9-8437-dad7ea15cae9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:21.419069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 682/1371] 10.222.0.16 () {54 vars in 964 bytes} [Sat Sep 17 20:57:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:21.423888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08fd4f-110a-489e-8130-04d392a01d11 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:21.424237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08fd4f-110a-489e-8130-04d392a01d11 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:21.424596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08fd4f-110a-489e-8130-04d392a01d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:21.504600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-dc08fd4f-110a-489e-8130-04d392a01d11 None None] MFA Rules not processed for user `3f6f37bc49c64ed1a9a61accef657525`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:21.505912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-dc08fd4f-110a-489e-8130-04d392a01d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:21.558151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.591270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 690/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Sep 17 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:21.599193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8522c7-ffab-40c9-8437-dad7ea15cae9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:21.601646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0f8522c7-ffab-40c9-8437-dad7ea15cae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:21.656311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.718621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8522c7-ffab-40c9-8437-dad7ea15cae9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:21.803363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:21.804269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:21.804553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:21.804864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:21.806454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:21.928442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.939975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:21.941520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b95bc-5fe2-4175-9120-f97742f2aea6 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:21.963489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 683/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:22.976947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d8552-3fa4-4e8c-aa04-c0e9d796a4ac None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:22.977514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d8552-3fa4-4e8c-aa04-c0e9d796a4ac None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:22.977514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d8552-3fa4-4e8c-aa04-c0e9d796a4ac None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:22.978871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 691/1374] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:57:22.984366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7dc0e-06f8-43ad-8b14-466c2fe1b865 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:22.984849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7dc0e-06f8-43ad-8b14-466c2fe1b865 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:22.985087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7dc0e-06f8-43ad-8b14-466c2fe1b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:23.051683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-eeb7dc0e-06f8-43ad-8b14-466c2fe1b865 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:23.052967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-eeb7dc0e-06f8-43ad-8b14-466c2fe1b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:23.149948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 684/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:23.443525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d8552-3fa4-4e8c-aa04-c0e9d796a4ac None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:23.445228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-514d8552-3fa4-4e8c-aa04-c0e9d796a4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:23.609047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d8552-3fa4-4e8c-aa04-c0e9d796a4ac None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:23.662125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:23.728108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:23.728735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:23.729011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:23.729281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:23.730878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:23.822671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:23.824279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b72a7-bbb4-490e-96d3-9c2e60c9390f service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:23.843660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 692/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:26.102336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630f0e-54b8-4e65-89d1-c3134add5604 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.102895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630f0e-54b8-4e65-89d1-c3134add5604 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.102895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91630f0e-54b8-4e65-89d1-c3134add5604 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.104388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 685/1377] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:26.109574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b9970-3ea8-4638-b98f-fa6d76c8edbe None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.109983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b9970-3ea8-4638-b98f-fa6d76c8edbe None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.110158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b9970-3ea8-4638-b98f-fa6d76c8edbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.180598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-df1b9970-3ea8-4638-b98f-fa6d76c8edbe None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:26.181810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-df1b9970-3ea8-4638-b98f-fa6d76c8edbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:26.189046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:26.284711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 693/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:26.581312 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630f0e-54b8-4e65-89d1-c3134add5604 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:26.583083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-91630f0e-54b8-4e65-89d1-c3134add5604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:26.647519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:26.736988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91630f0e-54b8-4e65-89d1-c3134add5604 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:26.855116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:26.855864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:26.856241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:26.856606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:26.858303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:26.879179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:26.956016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:26.957917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa71928-4793-41d4-ae60-fb87685bc8eb service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:26.977405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 686/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:29.188561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473ab38-bb2c-4457-9845-e70f1d704691 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:29.189193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473ab38-bb2c-4457-9845-e70f1d704691 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:29.189193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473ab38-bb2c-4457-9845-e70f1d704691 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:29.190636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 694/1380] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:29.196478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463b129-c07e-4661-9ad8-032f0a34601c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:29.196931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463b129-c07e-4661-9ad8-032f0a34601c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:29.197063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463b129-c07e-4661-9ad8-032f0a34601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:29.265235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5463b129-c07e-4661-9ad8-032f0a34601c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:29.266448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5463b129-c07e-4661-9ad8-032f0a34601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:29.362738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:29.376896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 687/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:29.666771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473ab38-bb2c-4457-9845-e70f1d704691 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:29.668380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6473ab38-bb2c-4457-9845-e70f1d704691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:29.824366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:29.835638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473ab38-bb2c-4457-9845-e70f1d704691 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:29.952205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:29.952838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:29.953112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:29.953391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:29.954975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:30.053890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:30.056298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14630bc9-c893-4529-b8c4-385955382e97 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:30.070886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:30.079954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 695/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:32.718158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658437-ca5b-4a08-b896-6863137b12e2 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.718726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658437-ca5b-4a08-b896-6863137b12e2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.718726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658437-ca5b-4a08-b896-6863137b12e2 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.720027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 688/1383] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:32.726248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eceef-ce9b-4fe5-875f-a26bb4ba0908 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:32.726713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eceef-ce9b-4fe5-875f-a26bb4ba0908 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:32.726889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eceef-ce9b-4fe5-875f-a26bb4ba0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:32.794981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-087eceef-ce9b-4fe5-875f-a26bb4ba0908 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:32.796199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-087eceef-ce9b-4fe5-875f-a26bb4ba0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:32.894491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 696/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:33.207568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53658437-ca5b-4a08-b896-6863137b12e2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:33.210516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-53658437-ca5b-4a08-b896-6863137b12e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:33.377005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53658437-ca5b-4a08-b896-6863137b12e2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:33.447477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:33.499839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:33.500437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:33.500743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:33.501022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:33.502650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:33.602401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:33.604000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9da2e-ac86-41c1-91e5-6750d685ef4b service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:33.622197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 689/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:36.107359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd87d8-85be-4f85-8150-c26d843c796d None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:36.107965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd87d8-85be-4f85-8150-c26d843c796d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:36.107965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd87d8-85be-4f85-8150-c26d843c796d None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:36.109288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 697/1386] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:36.114508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01170-62d8-46be-b7b3-d0b471dcdc59 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:36.115106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01170-62d8-46be-b7b3-d0b471dcdc59 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:36.115106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01170-62d8-46be-b7b3-d0b471dcdc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:36.180897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-aaf01170-62d8-46be-b7b3-d0b471dcdc59 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:36.182111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-aaf01170-62d8-46be-b7b3-d0b471dcdc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:36.215177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:36.278489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 690/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:36.569352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd87d8-85be-4f85-8150-c26d843c796d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:36.570885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a0bd87d8-85be-4f85-8150-c26d843c796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:36.669624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:36.739193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd87d8-85be-4f85-8150-c26d843c796d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:36.851735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:36.852968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:36.853243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:36.853514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:36.855111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:36.893672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:36.946166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:36.947772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f01c5-1162-43ad-933e-46090c601f80 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:36.966228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 698/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:37.298032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6393d-f6cf-426e-800c-1302b731e6bc None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:37.298568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6393d-f6cf-426e-800c-1302b731e6bc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:37.298568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6393d-f6cf-426e-800c-1302b731e6bc None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:37.300091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 691/1389] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:37.306844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e084a-edfc-49f5-ab0d-14ca7322ace9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:37.307465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e084a-edfc-49f5-ab0d-14ca7322ace9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:37.307779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e084a-edfc-49f5-ab0d-14ca7322ace9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:37.377590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d50e084a-edfc-49f5-ab0d-14ca7322ace9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:37.378820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d50e084a-edfc-49f5-ab0d-14ca7322ace9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:37.491820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 699/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:37.804445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1745adc-a3a5-4d16-8b91-df12621e5ba9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:37.804832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1745adc-a3a5-4d16-8b91-df12621e5ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:37.805202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1745adc-a3a5-4d16-8b91-df12621e5ba9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:37.806602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 692/1391] 10.222.0.16 () {54 vars in 962 bytes} [Sat Sep 17 20:57:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:37.813079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e3335-8bb2-4d53-840b-9b93977c9fbe None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:37.813369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e3335-8bb2-4d53-840b-9b93977c9fbe None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:37.813643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e3335-8bb2-4d53-840b-9b93977c9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:37.830474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:37.899512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-bf8e3335-8bb2-4d53-840b-9b93977c9fbe None None] MFA Rules not processed for user `82ecac0d9cc54fefbca5e3b6d81c31b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:37.900743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-bf8e3335-8bb2-4d53-840b-9b93977c9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:37.984626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 700/1392] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 17 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:57:37.994394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1745adc-a3a5-4d16-8b91-df12621e5ba9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:37.995921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f1745adc-a3a5-4d16-8b91-df12621e5ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:38.110601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1745adc-a3a5-4d16-8b91-df12621e5ba9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:38.174686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:38.191249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:38.191877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:38.192159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:38.192506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:38.195688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:38.332971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:38.334792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50aaa6-c606-454b-ad60-85762642b474 service glance] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:38.353584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 693/1393] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:39.109178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68d8b-a92a-442d-8663-c3b88c65e014 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:39.109649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68d8b-a92a-442d-8663-c3b88c65e014 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:39.109740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68d8b-a92a-442d-8663-c3b88c65e014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:39.179960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4cd68d8b-a92a-442d-8663-c3b88c65e014 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:39.181213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4cd68d8b-a92a-442d-8663-c3b88c65e014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:39.238407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:39.272477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 701/1394] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:39.319397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45aca5-fb63-4e88-9efe-dfe8eb14e6ad None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:39.319706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45aca5-fb63-4e88-9efe-dfe8eb14e6ad None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:39.319977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45aca5-fb63-4e88-9efe-dfe8eb14e6ad None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:39.321371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 694/1395] 10.222.0.16 () {54 vars in 943 bytes} [Sat Sep 17 20:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:39.327930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:39.328221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:39.328496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:39.340371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84f6e-4dd1-4119-aea6-1e2d11133440 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:39.340874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84f6e-4dd1-4119-aea6-1e2d11133440 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:39.340986 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd84f6e-4dd1-4119-aea6-1e2d11133440 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:39.342303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 695/1396] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:57:39.347306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:39.347590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:39.347869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:39.412581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] MFA Rules not processed for user `67e76f823b3a48368f5476c36acb9341`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:39.413820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:39.417527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:39.418697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:39.461646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:39.519907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 696/1397] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:57:39.526010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 702/1398] 10.222.0.16 () {58 vars in 1040 bytes} [Sat Sep 17 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:39.535550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:39.537349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:39.639041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2087079-eda1-4965-84e3-0e5a138b2086 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:39.701813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:39.741790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:39.742392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:39.742666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:39.742935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:39.744539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:39.809337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:39.810548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:39.830033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:39.831653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a136e3-8d2f-416e-b5eb-26e456acecaf service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:39.840207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 697/1399] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 20:57:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 20:57:39.844708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:39.921114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea5957-bda4-4fb0-9f79-6e6f790d661b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:40.072268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:40.072909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:40.073182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:40.073479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:40.075090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:40.128494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:40.183488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:40.185088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228d8d3-ef87-48f5-87c1-d11591e8b29e service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:40.205592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 703/1400] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:57:42.420151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e949-43fd-4e10-acef-1ba0156ca19d None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:42.420864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e949-43fd-4e10-acef-1ba0156ca19d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:42.420864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6e949-43fd-4e10-acef-1ba0156ca19d None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:42.422217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 698/1401] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:42.427802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868077bc-9f42-43a1-95d0-b75d46fca0f3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:42.428218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868077bc-9f42-43a1-95d0-b75d46fca0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:42.428370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868077bc-9f42-43a1-95d0-b75d46fca0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:42.497012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-868077bc-9f42-43a1-95d0-b75d46fca0f3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:42.498224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-868077bc-9f42-43a1-95d0-b75d46fca0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:42.604739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 704/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:42.896155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e949-43fd-4e10-acef-1ba0156ca19d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:42.898759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f7e6e949-43fd-4e10-acef-1ba0156ca19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:43.007561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:43.063996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6e949-43fd-4e10-acef-1ba0156ca19d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:43.203820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:43.204490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:43.204938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:43.205217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:43.206861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:43.278050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:43.311796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:43.313779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aebb4f-00c4-4458-b34c-51c6cd2377a2 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:43.337701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 699/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:45.629771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898211e6-f759-425a-a15d-e6a42629c865 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:45.631142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898211e6-f759-425a-a15d-e6a42629c865 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:45.631142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898211e6-f759-425a-a15d-e6a42629c865 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:45.632705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 705/1404] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:45.638124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3c194-00e9-44b0-aff5-b939fc747b37 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:45.638552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3c194-00e9-44b0-aff5-b939fc747b37 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:45.640477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df3c194-00e9-44b0-aff5-b939fc747b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:45.712219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-9df3c194-00e9-44b0-aff5-b939fc747b37 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:45.713473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9df3c194-00e9-44b0-aff5-b939fc747b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:45.820429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 700/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:57:46.115218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898211e6-f759-425a-a15d-e6a42629c865 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:46.116788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-898211e6-f759-425a-a15d-e6a42629c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:46.138302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:46.288985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898211e6-f759-425a-a15d-e6a42629c865 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:46.393830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:46.419211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:46.419978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:46.420320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:46.420604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:46.422244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:46.524364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:46.526082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed13d1-e4a8-4494-b968-4b1e7e08f865 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:46.546380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 706/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:48.769893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4263c1b-1dbe-4afe-937b-f6d29be65095 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:48.770486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4263c1b-1dbe-4afe-937b-f6d29be65095 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:48.770486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4263c1b-1dbe-4afe-937b-f6d29be65095 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:48.772032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 701/1407] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:57:48.777502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c607078-46dd-4b19-adff-7b9895505352 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:48.777961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c607078-46dd-4b19-adff-7b9895505352 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:48.778077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c607078-46dd-4b19-adff-7b9895505352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:48.846550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7c607078-46dd-4b19-adff-7b9895505352 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:57:48.847783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7c607078-46dd-4b19-adff-7b9895505352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:48.909734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:48.958401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 707/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:57:49.252172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4263c1b-1dbe-4afe-937b-f6d29be65095 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:57:49.253780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d4263c1b-1dbe-4afe-937b-f6d29be65095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:49.303464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:49.421481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4263c1b-1dbe-4afe-937b-f6d29be65095 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:57:49.547033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:57:49.547561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:57:49.547845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:57:49.548119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:57:49.549772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:57:49.552428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:49.650803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:57:49.652435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b61c75-3483-4725-8fb9-e6a7a21b2053 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:57:49.671281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 702/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:57:58.956488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=19 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:59.669030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=44 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:02.797346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b2e4b-9fee-4e0b-9b36-bb4f17b9a169 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:02.797916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b2e4b-9fee-4e0b-9b36-bb4f17b9a169 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:02.797916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b2e4b-9fee-4e0b-9b36-bb4f17b9a169 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:02.799272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 708/1410] 10.222.0.16 () {54 vars in 918 bytes} [Sat Sep 17 20:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:02.804534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1071180-e25a-4555-b0fe-abcc8b9293e7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:02.804981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1071180-e25a-4555-b0fe-abcc8b9293e7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:02.805109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1071180-e25a-4555-b0fe-abcc8b9293e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:02.889794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b1071180-e25a-4555-b0fe-abcc8b9293e7 None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:02.890997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b1071180-e25a-4555-b0fe-abcc8b9293e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:02.996444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 703/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Sep 17 20:58:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:03.010944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ecf1-d248-4a46-bcf3-7a3f85bee9f9 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.011225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ecf1-d248-4a46-bcf3-7a3f85bee9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.011533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773ecf1-d248-4a46-bcf3-7a3f85bee9f9 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.012914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 709/1412] 10.222.0.16 () {54 vars in 918 bytes} [Sat Sep 17 20:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:03.017438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda9110-4e74-4641-b430-449bad30ffc0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:03.017722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda9110-4e74-4641-b430-449bad30ffc0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:03.017995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda9110-4e74-4641-b430-449bad30ffc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:03.092668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:03.095454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0fda9110-4e74-4641-b430-449bad30ffc0 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:03.097040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0fda9110-4e74-4641-b430-449bad30ffc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:03.214137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 704/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Sep 17 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:05.809970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484df73c-1bda-4632-afab-2b41098293f3 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:05.810531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484df73c-1bda-4632-afab-2b41098293f3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:05.810531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484df73c-1bda-4632-afab-2b41098293f3 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:05.811885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:05.818116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428f0a5-5c2b-43c0-b5d9-ae07094beb4b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:05.818536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428f0a5-5c2b-43c0-b5d9-ae07094beb4b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:05.818650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428f0a5-5c2b-43c0-b5d9-ae07094beb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:05.884908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0428f0a5-5c2b-43c0-b5d9-ae07094beb4b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:05.886118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0428f0a5-5c2b-43c0-b5d9-ae07094beb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:05.939736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:05.983090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:06.273314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484df73c-1bda-4632-afab-2b41098293f3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:06.274922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-484df73c-1bda-4632-afab-2b41098293f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:06.418224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484df73c-1bda-4632-afab-2b41098293f3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:06.489180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:06.529814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:06.530369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:06.530637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:06.530901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:06.532465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:06.620959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:06.622559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232aacef-759c-419c-adcc-8c6a26df3a57 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:06.640635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 711/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:07.777326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d642a7-5253-4ec2-ab1a-54f1a4cdd2ce None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:07.777902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d642a7-5253-4ec2-ab1a-54f1a4cdd2ce None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:07.777902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d642a7-5253-4ec2-ab1a-54f1a4cdd2ce None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:07.779313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:07.786310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b43aa-f229-4aef-afed-9c4ed9475d2d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:07.786754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b43aa-f229-4aef-afed-9c4ed9475d2d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:07.786861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b43aa-f229-4aef-afed-9c4ed9475d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:07.861829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-519b43aa-f229-4aef-afed-9c4ed9475d2d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:07.863033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-519b43aa-f229-4aef-afed-9c4ed9475d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:07.889311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:07.959017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:08.107233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d642a7-5253-4ec2-ab1a-54f1a4cdd2ce None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:08.108661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b6d642a7-5253-4ec2-ab1a-54f1a4cdd2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:08.251858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d642a7-5253-4ec2-ab1a-54f1a4cdd2ce None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:08.292264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:08.362806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:08.363333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:08.363699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:08.363972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:08.365686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:08.456016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:08.458218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db8be0b-b886-44dc-967a-84b73f1a724b service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:08.475665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 707/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:09.607660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975ced4-85f1-4bce-9976-3e7b582455fa None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:09.608227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975ced4-85f1-4bce-9976-3e7b582455fa None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:09.608227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975ced4-85f1-4bce-9976-3e7b582455fa None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:09.609595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:09.616548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff3a41-d781-4161-bb7e-cb6ad08f7010 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:09.617008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff3a41-d781-4161-bb7e-cb6ad08f7010 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:09.617151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff3a41-d781-4161-bb7e-cb6ad08f7010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:09.670176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:09.693667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-f7ff3a41-d781-4161-bb7e-cb6ad08f7010 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:09.694874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f7ff3a41-d781-4161-bb7e-cb6ad08f7010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:09.790744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:09.940053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975ced4-85f1-4bce-9976-3e7b582455fa None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:09.941743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c975ced4-85f1-4bce-9976-3e7b582455fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:10.090889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975ced4-85f1-4bce-9976-3e7b582455fa None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:10.103459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:10.203042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:10.203554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:10.203824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:10.204091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:10.205698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:10.292589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:10.294198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1754dc-acc7-4128-bf78-a4a4c6e160bd service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:10.311688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 714/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:11.424651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f98ea-0fc1-4796-97a1-2316afd52a39 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.425189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f98ea-0fc1-4796-97a1-2316afd52a39 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.425294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f98ea-0fc1-4796-97a1-2316afd52a39 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.427225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:11.433641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2066395-8a34-41f8-9c2b-43f25e6b47ab None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:11.434111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2066395-8a34-41f8-9c2b-43f25e6b47ab None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:11.434230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2066395-8a34-41f8-9c2b-43f25e6b47ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:11.454407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:11.510895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f2066395-8a34-41f8-9c2b-43f25e6b47ab None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:11.512099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f2066395-8a34-41f8-9c2b-43f25e6b47ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:11.608555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:11.756974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f98ea-0fc1-4796-97a1-2316afd52a39 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:11.758490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-253f98ea-0fc1-4796-97a1-2316afd52a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:11.867408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:11.903265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f98ea-0fc1-4796-97a1-2316afd52a39 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:12.014223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:12.014752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:12.015032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:12.015306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:12.016961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:12.093087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:12.106582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:12.108191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f94499-9cc3-4be2-85fd-13af07188922 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:12.125421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 710/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:13.232578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3475bc0-287b-4f91-b879-d0a894981355 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:13.233180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3475bc0-287b-4f91-b879-d0a894981355 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:13.233180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3475bc0-287b-4f91-b879-d0a894981355 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:13.234500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:13.241661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f556e-f284-454b-849b-75deaa6e2090 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:13.242091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f556e-f284-454b-849b-75deaa6e2090 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:13.242234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f556e-f284-454b-849b-75deaa6e2090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:13.318173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-065f556e-f284-454b-849b-75deaa6e2090 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:13.319396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-065f556e-f284-454b-849b-75deaa6e2090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:13.415744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:13.566304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3475bc0-287b-4f91-b879-d0a894981355 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:13.567926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a3475bc0-287b-4f91-b879-d0a894981355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:13.650989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:13.721211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3475bc0-287b-4f91-b879-d0a894981355 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:13.831370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:13.831893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:13.832681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:13.832980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:13.834654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:13.882343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:13.923880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:13.925503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de2d47-2641-4415-9d67-43ce0983b800 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:13.943128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 717/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:15.003352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e379fd0-cf5b-401a-b36c-a976ba90d970 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:15.003893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e379fd0-cf5b-401a-b36c-a976ba90d970 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:15.004018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e379fd0-cf5b-401a-b36c-a976ba90d970 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:15.005719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:15.012107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc081-6101-4183-b04d-9fa02b16648c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:15.012542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc081-6101-4183-b04d-9fa02b16648c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:15.012654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc081-6101-4183-b04d-9fa02b16648c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:15.088828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-843dc081-6101-4183-b04d-9fa02b16648c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:15.090028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-843dc081-6101-4183-b04d-9fa02b16648c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:15.185809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:15.335424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e379fd0-cf5b-401a-b36c-a976ba90d970 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:15.337029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5e379fd0-cf5b-401a-b36c-a976ba90d970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:15.384935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:15.480920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e379fd0-cf5b-401a-b36c-a976ba90d970 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:15.590397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:15.590999 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:15.591277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:15.591550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:15.593167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:15.600582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:15.683107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:15.684711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960104a-c48d-47a0-b01d-1cd3ec595d98 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:15.702916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 713/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:16.760971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e574c6-50b0-4f31-99aa-b84417e01eb4 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:16.761534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e574c6-50b0-4f31-99aa-b84417e01eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:16.761534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e574c6-50b0-4f31-99aa-b84417e01eb4 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:16.762863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:16.768955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf11ed-2441-4c97-884d-9c5ab26ada0d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:16.769360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf11ed-2441-4c97-884d-9c5ab26ada0d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:16.769506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf11ed-2441-4c97-884d-9c5ab26ada0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:16.844285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5ddf11ed-2441-4c97-884d-9c5ab26ada0d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:16.845808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5ddf11ed-2441-4c97-884d-9c5ab26ada0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:16.914399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:16.943538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:17.093436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e574c6-50b0-4f31-99aa-b84417e01eb4 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:17.094997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a8e574c6-50b0-4f31-99aa-b84417e01eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:17.111937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:17.239540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e574c6-50b0-4f31-99aa-b84417e01eb4 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:17.324191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:17.350165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:17.350694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:17.350968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:17.351512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:17.353134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:17.445896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:17.447482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58149b-31a7-4314-a741-82e8fee2835c service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:17.465377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 720/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:18.523649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62c198-dbe7-4a6d-b2fe-5f2829ef6b77 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:18.524239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62c198-dbe7-4a6d-b2fe-5f2829ef6b77 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:18.524239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62c198-dbe7-4a6d-b2fe-5f2829ef6b77 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:18.525634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:18.531846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefd96-27da-454b-9e4b-5c427026a6cb None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:18.532282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefd96-27da-454b-9e4b-5c427026a6cb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:18.532410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eefd96-27da-454b-9e4b-5c427026a6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:18.614301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-05eefd96-27da-454b-9e4b-5c427026a6cb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:18.615534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-05eefd96-27da-454b-9e4b-5c427026a6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:18.663351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:18.719369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:18.869212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62c198-dbe7-4a6d-b2fe-5f2829ef6b77 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:18.870785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9f62c198-dbe7-4a6d-b2fe-5f2829ef6b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:19.017003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62c198-dbe7-4a6d-b2fe-5f2829ef6b77 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:19.072099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:19.131071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:19.131676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:19.131950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:19.132218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:19.133859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:19.222064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:19.223772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcf3e6-5065-4cf1-a506-ebbd715b0956 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:19.243098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 716/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:20.310328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76447d-46b7-479a-8b24-dd6ad4b021cf None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:20.310934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76447d-46b7-479a-8b24-dd6ad4b021cf None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:20.310934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76447d-46b7-479a-8b24-dd6ad4b021cf None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:20.312270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 20:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:20.319202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e980e43-df2e-4a06-8d99-70e2d316825a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:20.319670 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e980e43-df2e-4a06-8d99-70e2d316825a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:20.319787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e980e43-df2e-4a06-8d99-70e2d316825a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:20.400341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8e980e43-df2e-4a06-8d99-70e2d316825a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:20.401609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8e980e43-df2e-4a06-8d99-70e2d316825a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:20.403615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:20.496218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:20.659708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76447d-46b7-479a-8b24-dd6ad4b021cf None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:20.661655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fd76447d-46b7-479a-8b24-dd6ad4b021cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:20.804546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76447d-46b7-479a-8b24-dd6ad4b021cf None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:20.830992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:20.915681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:20.916203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:20.916481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:20.916782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:20.918403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:21.008248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:21.009899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd7b73-3a8d-4030-8b73-95c7e67b2cff service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:21.027631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 723/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:23.070358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357782-74db-40bc-bf9e-aa3d60881568 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:23.071069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357782-74db-40bc-bf9e-aa3d60881568 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:23.071069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13357782-74db-40bc-bf9e-aa3d60881568 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:23.072313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:58:23.078092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9bc16-904b-4e9b-aff6-f2eeefd96503 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:23.078523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9bc16-904b-4e9b-aff6-f2eeefd96503 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:23.078681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9bc16-904b-4e9b-aff6-f2eeefd96503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:23.120808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:23.146316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d9f9bc16-904b-4e9b-aff6-f2eeefd96503 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:23.147532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d9f9bc16-904b-4e9b-aff6-f2eeefd96503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:23.243964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:23.530877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357782-74db-40bc-bf9e-aa3d60881568 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:23.532419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-13357782-74db-40bc-bf9e-aa3d60881568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:23.634918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13357782-74db-40bc-bf9e-aa3d60881568 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:23.666070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:23.707639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:23.708240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:23.708520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:23.708821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:23.710407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:23.817112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:23.818553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c4971-3670-4fc6-b19e-235ce5344a00 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:23.836281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 719/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:26.205679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818185-d3f5-4c24-8a70-b5f30ac4d6b3 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:26.206254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818185-d3f5-4c24-8a70-b5f30ac4d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:26.206254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818185-d3f5-4c24-8a70-b5f30ac4d6b3 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:26.208361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:26.213685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d62f3-8e70-464a-add7-d7fccdc879f0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:26.214104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d62f3-8e70-464a-add7-d7fccdc879f0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:26.214233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d62f3-8e70-464a-add7-d7fccdc879f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:26.280194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ac0d62f3-8e70-464a-add7-d7fccdc879f0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:26.281458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ac0d62f3-8e70-464a-add7-d7fccdc879f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:26.294246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:26.379609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:26.668373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab818185-d3f5-4c24-8a70-b5f30ac4d6b3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:26.669937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ab818185-d3f5-4c24-8a70-b5f30ac4d6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:26.772377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab818185-d3f5-4c24-8a70-b5f30ac4d6b3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:26.785350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:26.846726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:26.847284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:26.847650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:26.847925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:26.849549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:26.957174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:26.958654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4236e-072c-4069-ae03-2865c5ca82ed service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:26.977016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 726/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:29.439603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df3440-8ae1-4a1e-851d-dd90229e057c None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.440163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df3440-8ae1-4a1e-851d-dd90229e057c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.440163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df3440-8ae1-4a1e-851d-dd90229e057c None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.442295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:29.447900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ef9a9-cef4-4d72-a845-9f6dec17da7f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:29.448419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ef9a9-cef4-4d72-a845-9f6dec17da7f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:29.448529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ef9a9-cef4-4d72-a845-9f6dec17da7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:29.500300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:29.518290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f45ef9a9-cef4-4d72-a845-9f6dec17da7f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:29.520128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f45ef9a9-cef4-4d72-a845-9f6dec17da7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:29.615638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:29.900923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df3440-8ae1-4a1e-851d-dd90229e057c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:29.902672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-11df3440-8ae1-4a1e-851d-dd90229e057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:30.004630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df3440-8ae1-4a1e-851d-dd90229e057c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:30.044946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:30.079558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:30.080183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:30.080459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:30.080756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:30.083042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:30.189059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:30.190495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72e1a8-1884-4c47-8213-a6e6828cc4c2 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:30.209918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 722/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:32.494675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad50173-1626-4f75-9dac-67b0cb3d8cce None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:32.495256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad50173-1626-4f75-9dac-67b0cb3d8cce None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:32.495256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad50173-1626-4f75-9dac-67b0cb3d8cce None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:32.496555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:32.501867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7e5ab-2cf2-4fee-89d4-4031b6544c37 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:32.502262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7e5ab-2cf2-4fee-89d4-4031b6544c37 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:32.502439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7e5ab-2cf2-4fee-89d4-4031b6544c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:32.568791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-cdf7e5ab-2cf2-4fee-89d4-4031b6544c37 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:32.569998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cdf7e5ab-2cf2-4fee-89d4-4031b6544c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:32.590145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:32.666116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:32.949872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad50173-1626-4f75-9dac-67b0cb3d8cce None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:32.951642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6ad50173-1626-4f75-9dac-67b0cb3d8cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:33.053596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad50173-1626-4f75-9dac-67b0cb3d8cce None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:33.072957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:33.127566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:33.128163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:33.128501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:33.128792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:33.130370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:33.237473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:33.239524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad72f4-e893-480f-a5fb-b01f53ea9f11 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:33.257185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 729/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:35.463627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6952d7-5804-4447-b8f0-f88a21b1b025 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:35.464148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6952d7-5804-4447-b8f0-f88a21b1b025 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:35.464263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6952d7-5804-4447-b8f0-f88a21b1b025 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:35.465582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:35.471056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38232f-d936-43bf-aca2-92fc5497eb9b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:35.471474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38232f-d936-43bf-aca2-92fc5497eb9b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:35.471585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38232f-d936-43bf-aca2-92fc5497eb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:35.534061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:35.536971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9d38232f-d936-43bf-aca2-92fc5497eb9b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:35.538778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9d38232f-d936-43bf-aca2-92fc5497eb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:35.634742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:35.916023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6952d7-5804-4447-b8f0-f88a21b1b025 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:35.917633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2f6952d7-5804-4447-b8f0-f88a21b1b025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:36.019719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6952d7-5804-4447-b8f0-f88a21b1b025 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:36.068684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:36.093731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:36.094336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:36.094619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:36.094891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:36.096502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:36.204721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:36.206165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df8f65-af75-4df4-8ebc-be7b134b0a1e service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:36.224313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 725/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:38.420802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca0936-58e2-47fa-bf78-d69fde4549c6 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:38.421324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca0936-58e2-47fa-bf78-d69fde4549c6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:38.421324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca0936-58e2-47fa-bf78-d69fde4549c6 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:38.422898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:38.432163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fab91b-1481-485a-bf19-45386c66d1ea None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:38.432662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fab91b-1481-485a-bf19-45386c66d1ea None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:38.432788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fab91b-1481-485a-bf19-45386c66d1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:38.504088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-30fab91b-1481-485a-bf19-45386c66d1ea None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:38.505352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-30fab91b-1481-485a-bf19-45386c66d1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:38.539415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:38.613595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:38.902617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca0936-58e2-47fa-bf78-d69fde4549c6 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:38.904138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9eca0936-58e2-47fa-bf78-d69fde4549c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:39.018569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca0936-58e2-47fa-bf78-d69fde4549c6 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:39.047913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:39.095806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:39.096404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:39.096704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:39.097225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:39.098844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:39.217671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:39.219194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70799e6-2101-46f7-998e-53401412c960 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:39.237167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 732/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:41.442310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c443a-7b00-436b-bff0-a624d66138b9 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:41.442838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c443a-7b00-436b-bff0-a624d66138b9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:41.442838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c443a-7b00-436b-bff0-a624d66138b9 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:41.444204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:41.449448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db981a-888d-490a-adce-38ef7f3bd996 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:41.449890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db981a-888d-490a-adce-38ef7f3bd996 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:41.450261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db981a-888d-490a-adce-38ef7f3bd996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:41.516949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-36db981a-888d-490a-adce-38ef7f3bd996 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:41.517838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-36db981a-888d-490a-adce-38ef7f3bd996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:41.526778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:41.616493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:41.898437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c443a-7b00-436b-bff0-a624d66138b9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:41.900033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5a4c443a-7b00-436b-bff0-a624d66138b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:42.002700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c443a-7b00-436b-bff0-a624d66138b9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:42.174502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:42.194327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:42.194934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:42.195216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:42.195484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:42.197389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:42.315141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:42.316966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd59a744-e5bc-4cc8-848d-656d79771827 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:42.336556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 728/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:44.561472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60974e7-04ee-4264-ad30-e99924c26698 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.562003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60974e7-04ee-4264-ad30-e99924c26698 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.562003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60974e7-04ee-4264-ad30-e99924c26698 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.563378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:44.569410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761b480-a8d7-4185-ae43-c951a11d4303 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:44.570110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761b480-a8d7-4185-ae43-c951a11d4303 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:44.570110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761b480-a8d7-4185-ae43-c951a11d4303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:44.636078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5761b480-a8d7-4185-ae43-c951a11d4303 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:44.637342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5761b480-a8d7-4185-ae43-c951a11d4303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:44.674521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:44.732662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:58:45.020287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60974e7-04ee-4264-ad30-e99924c26698 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:45.021870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c60974e7-04ee-4264-ad30-e99924c26698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:45.124814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60974e7-04ee-4264-ad30-e99924c26698 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:45.163169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:45.201222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:45.202036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:45.202181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:45.202449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:45.204134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:45.313477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:45.314921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ba843c-1b73-4fe1-8145-def10bfd5368 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:45.333565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 735/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:47.515054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026b72-7b57-4056-bff6-e60784624061 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.515605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026b72-7b57-4056-bff6-e60784624061 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.515605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05026b72-7b57-4056-bff6-e60784624061 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.517066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:47.522908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac5fb-c287-40ca-95a6-40d22ea751cf None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:47.523312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac5fb-c287-40ca-95a6-40d22ea751cf None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:47.523454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac5fb-c287-40ca-95a6-40d22ea751cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:47.589834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6a8ac5fb-c287-40ca-95a6-40d22ea751cf None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:47.591037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6a8ac5fb-c287-40ca-95a6-40d22ea751cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:47.609404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:47.685723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:47.974368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05026b72-7b57-4056-bff6-e60784624061 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:47.975894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-05026b72-7b57-4056-bff6-e60784624061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:48.078634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05026b72-7b57-4056-bff6-e60784624061 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:48.142898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:48.150495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:48.151089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:48.151361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:48.151626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:48.153228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:48.260762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:48.262137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0adb91-f251-4d21-9ee6-9a9757da82a9 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:48.280272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 731/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:58:50.463642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652281c-e6f7-4069-9e82-2cdcfc53967f None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.464222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652281c-e6f7-4069-9e82-2cdcfc53967f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.464222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652281c-e6f7-4069-9e82-2cdcfc53967f None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.465581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 20:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:58:50.472806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7f2b3-9b00-4f08-8e9a-31b0bfb9d812 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:50.473306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7f2b3-9b00-4f08-8e9a-31b0bfb9d812 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:50.473698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7f2b3-9b00-4f08-8e9a-31b0bfb9d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:50.538913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-63f7f2b3-9b00-4f08-8e9a-31b0bfb9d812 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:58:50.540124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-63f7f2b3-9b00-4f08-8e9a-31b0bfb9d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:50.588641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:50.638041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:58:50.923061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652281c-e6f7-4069-9e82-2cdcfc53967f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:58:50.925273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a652281c-e6f7-4069-9e82-2cdcfc53967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.028007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652281c-e6f7-4069-9e82-2cdcfc53967f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:58:51.078685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:51.107423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:58:51.108055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:58:51.108329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:58:51.108599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:58:51.110285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:58:51.223306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:58:51.224760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6855c9f-0264-43f7-a0a6-70c62c3ce468 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:58:51.243188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 738/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:58:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:00.636494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=22 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:01.241006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=66 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:11.191758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c0dac-9382-4b83-8c3a-4410358fb9b8 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.192297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c0dac-9382-4b83-8c3a-4410358fb9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.192297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c0dac-9382-4b83-8c3a-4410358fb9b8 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.193697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 733/1471] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 20:59:11.199105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1838-0461-490a-ac87-8fca7f143b7d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.199519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1838-0461-490a-ac87-8fca7f143b7d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.199651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f1838-0461-490a-ac87-8fca7f143b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.264819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1f9f1838-0461-490a-ac87-8fca7f143b7d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:11.266011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1f9f1838-0461-490a-ac87-8fca7f143b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:11.360328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 739/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:11.658879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19c11-8302-4fad-a176-e545da832142 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.659164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19c11-8302-4fad-a176-e545da832142 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.659437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19c11-8302-4fad-a176-e545da832142 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.660794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 734/1473] 10.222.0.16 () {54 vars in 962 bytes} [Sat Sep 17 20:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:11.665875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bf99f-8b4b-4cd7-9cee-20f7a7d56dda None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:11.666159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bf99f-8b4b-4cd7-9cee-20f7a7d56dda None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:11.666435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0bf99f-8b4b-4cd7-9cee-20f7a7d56dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:11.741997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0a0bf99f-8b4b-4cd7-9cee-20f7a7d56dda None None] MFA Rules not processed for user `82ecac0d9cc54fefbca5e3b6d81c31b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:11.743202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0a0bf99f-8b4b-4cd7-9cee-20f7a7d56dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:11.763088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:11.817393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 740/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Sat Sep 17 20:59:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:11.825261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19c11-8302-4fad-a176-e545da832142 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:11.826770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8ac19c11-8302-4fad-a176-e545da832142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:11.931030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19c11-8302-4fad-a176-e545da832142 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:12.004450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:12.005072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:12.005350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:12.005617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:12.007187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:12.060518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:12.117335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:12.118779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121d921-ab4b-4f27-b933-266a127672e8 service glance] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:12.137680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 735/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 20:59:14.375403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4af46-6549-46ad-bdb8-7d757afe365c None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:14.375975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4af46-6549-46ad-bdb8-7d757afe365c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:14.375975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4af46-6549-46ad-bdb8-7d757afe365c None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:14.377407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 741/1476] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:14.382881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c41d0d-3c36-452a-8da8-2a4af2bd687b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:14.383326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c41d0d-3c36-452a-8da8-2a4af2bd687b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:14.383431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c41d0d-3c36-452a-8da8-2a4af2bd687b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:14.450624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-81c41d0d-3c36-452a-8da8-2a4af2bd687b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:14.451834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-81c41d0d-3c36-452a-8da8-2a4af2bd687b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:14.545904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 736/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:14.823723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4af46-6549-46ad-bdb8-7d757afe365c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:14.825580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8be4af46-6549-46ad-bdb8-7d757afe365c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:14.957118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4af46-6549-46ad-bdb8-7d757afe365c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:15.010856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:15.054398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:15.055014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:15.055286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:15.055551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:15.056175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:15.057986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 20:59:15.057986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 20:59:15.057986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 20:59:15.059133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd82bde-f14c-4d76-ab6b-87a4359b77cc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:15.078636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 742/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Sat Sep 17 20:59:14 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 20:59:17.262047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12fe5-ee1b-4407-939a-a1a25075c859 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:17.262664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12fe5-ee1b-4407-939a-a1a25075c859 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:17.262664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac12fe5-ee1b-4407-939a-a1a25075c859 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:17.264050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 737/1479] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:17.269311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909afa-00c9-4180-b381-1a5a619791f7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:17.269734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909afa-00c9-4180-b381-1a5a619791f7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:17.269860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74909afa-00c9-4180-b381-1a5a619791f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:17.335202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-74909afa-00c9-4180-b381-1a5a619791f7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:17.336446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-74909afa-00c9-4180-b381-1a5a619791f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:17.432689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 743/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:17.729605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12fe5-ee1b-4407-939a-a1a25075c859 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:17.731363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3ac12fe5-ee1b-4407-939a-a1a25075c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:17.741529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:17.878126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac12fe5-ee1b-4407-939a-a1a25075c859 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:17.957166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:17.988642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:17.989200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:17.989476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:17.989745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:17.991321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:18.081184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:18.082780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50761622-dde9-466f-b9a3-379d59531235 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:18.100273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 738/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:20.456657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312f51-20a0-40aa-be88-51cc808f548c None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:20.457243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312f51-20a0-40aa-be88-51cc808f548c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:20.457243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00312f51-20a0-40aa-be88-51cc808f548c None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:20.458573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 744/1482] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:20.463920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3ed37-600a-4c4a-bd4c-13855861e77e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:20.464353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3ed37-600a-4c4a-bd4c-13855861e77e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:20.464446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3ed37-600a-4c4a-bd4c-13855861e77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:20.529356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-38d3ed37-600a-4c4a-bd4c-13855861e77e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:20.530557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-38d3ed37-600a-4c4a-bd4c-13855861e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:20.577225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:20.626454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 739/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 20:59:20.914062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312f51-20a0-40aa-be88-51cc808f548c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:20.915622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-00312f51-20a0-40aa-be88-51cc808f548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:20.939954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:21.017866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00312f51-20a0-40aa-be88-51cc808f548c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:21.093605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:21.094199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:21.094471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:21.094735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:21.096891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:21.173044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:21.208062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:21.209522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858d86b-87b6-4a3c-852a-51880d310e07 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:21.227474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 745/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:23.433027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cf28-b986-44cf-b3f8-7576c4696469 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:23.433544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cf28-b986-44cf-b3f8-7576c4696469 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:23.433638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2cf28-b986-44cf-b3f8-7576c4696469 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:23.434925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 740/1485] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:23.441134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e563d-2ac4-45d6-bb93-f5da4a923fbb None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:23.441551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e563d-2ac4-45d6-bb93-f5da4a923fbb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:23.441677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e563d-2ac4-45d6-bb93-f5da4a923fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:23.506689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4d6e563d-2ac4-45d6-bb93-f5da4a923fbb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:23.507890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4d6e563d-2ac4-45d6-bb93-f5da4a923fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:23.604150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 746/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:23.896025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2cf28-b986-44cf-b3f8-7576c4696469 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:23.897574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-eda2cf28-b986-44cf-b3f8-7576c4696469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:24.001013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2cf28-b986-44cf-b3f8-7576c4696469 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:24.071960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:24.073564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:24.074207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:24.074485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:24.075062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:24.077202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:24.186159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:24.187540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fed43-a572-4162-bb5d-fdf1e6acf73f service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:24.205836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 741/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:26.529180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e33e-9cd2-4754-86be-8e563f6f4411 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:26.529728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e33e-9cd2-4754-86be-8e563f6f4411 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:26.529728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9e33e-9cd2-4754-86be-8e563f6f4411 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:26.531107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 747/1488] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:26.536891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5e29-041d-4a81-846e-00245ae80f3d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:26.537341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5e29-041d-4a81-846e-00245ae80f3d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:26.537450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d5e29-041d-4a81-846e-00245ae80f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:26.601810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5b7d5e29-041d-4a81-846e-00245ae80f3d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:26.603004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b7d5e29-041d-4a81-846e-00245ae80f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:26.654637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:26.698245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 742/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:26.989314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9e33e-9cd2-4754-86be-8e563f6f4411 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:26.990899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7ee9e33e-9cd2-4754-86be-8e563f6f4411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:27.026236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:27.093923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9e33e-9cd2-4754-86be-8e563f6f4411 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:27.169554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:27.170154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:27.170432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:27.170721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:27.172311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:27.252054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:27.280745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:27.282196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da79ded-b8d5-4519-922e-885468095e7b service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:27.301990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 748/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:29.664426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951c99-13c0-46b8-ac67-003f3a7e395b None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:29.664987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951c99-13c0-46b8-ac67-003f3a7e395b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:29.664987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951c99-13c0-46b8-ac67-003f3a7e395b None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:29.666250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 743/1491] 10.222.0.16 () {54 vars in 927 bytes} [Sat Sep 17 20:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 20:59:29.671564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196ef10-88f0-416a-a1c2-911161bc4652 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:29.671964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196ef10-88f0-416a-a1c2-911161bc4652 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:29.672126 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196ef10-88f0-416a-a1c2-911161bc4652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:29.737605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9196ef10-88f0-416a-a1c2-911161bc4652 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 20:59:29.738816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9196ef10-88f0-416a-a1c2-911161bc4652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:29.835487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 749/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Sat Sep 17 20:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 20:59:30.128262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951c99-13c0-46b8-ac67-003f3a7e395b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 20:59:30.129839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2d951c99-13c0-46b8-ac67-003f3a7e395b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:30.232467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951c99-13c0-46b8-ac67-003f3a7e395b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 20:59:30.306260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 20:59:30.306858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 20:59:30.307133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 20:59:30.307401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 20:59:30.309016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 20:59:30.318227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:30.418402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 20:59:30.419830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33566ad7-37cf-40ab-8908-4c3054217fc9 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 20:59:30.438145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 744/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 20:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 20:59:39.833986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=82 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:40.435947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=50 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:43.271663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:43.272250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:43.272250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:43.348570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:43.349810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:43.446991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 750/1494] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:43.455969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:43.456482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:43.456589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:43.535021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:43.536176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:43.630143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 745/1495] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:43.634757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:43.635991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:43.647673 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c28f-58f5-4abb-858c-21c961f84aca None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:43.825957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:43.859089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:43.859672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:43.859940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:43.860201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:43.860855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:00:43.861149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:43.862151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:43.862151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:00:43.862151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:00:43.863019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:43.874441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 751/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:00:43.882902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:43.892298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03af8fc-8c41-4766-af4f-09a527f37cc1 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:43.952148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:43.985237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:43.985831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:43.986107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:43.986372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:43.986992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:43.988713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:43.988713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:00:43.988713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:00:43.989914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:44.044617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 746/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:00:43 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:00:44.049582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:44.051073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:44.084047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638926a1-2c03-42df-809d-9ed88a33bd13 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:44.174129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:44.174728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.175006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.175278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.175916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:44.176991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:44.176991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:00:44.176991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:00:44.178214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:44.221119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 752/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:00:44 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:00:44.225402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:44.226907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:44.286618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87846d5b-ea9d-4739-a807-c91b0fb51079 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:44.370145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:44.381822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:44.382422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.382702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.382970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.383601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:44.384611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:44.384611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:00:44.384611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:00:44.385566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:44.398982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 747/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:00:44.403123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:44.405584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:44.416924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fdb5f-3d62-4640-82c4-0bbca3438ec2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:44.442249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:44.511475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:44.512071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.512339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.512604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.513256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:44.514534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:44.514534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:00:44.514534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:00:44.515412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c555f-6c87-42ca-b102-debdfeba6ea1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:44.528066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 753/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:00:44.532847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:44.534351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:44.567250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84a603-b37d-43e7-93ce-5f40e8fdad77 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:44.659018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:44.659589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.659865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.660135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] PATH_INFO: `/v3/projects/d728b386858c43ed9fce0158ec5d53a1/users/56fb92e8e3f542c197d697cbce709cae/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.682029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d728b386858c43ed9fce0158ec5d53a1, user_id=56fb92e8e3f542c197d697cbce709cae, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:44.683049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:44.683049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:00:44.683049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:00:44.685044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b4338-98c8-48fd-a000-6732852f29ca admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:44.709568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 748/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:00:44 2022] PUT /identity/v3/projects/d728b386858c43ed9fce0158ec5d53a1/users/56fb92e8e3f542c197d697cbce709cae/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:44.722137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb88d-ceb6-4a3e-84d5-05eb1a97bd19 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.722425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb88d-ceb6-4a3e-84d5-05eb1a97bd19 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.722703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eb88d-ceb6-4a3e-84d5-05eb1a97bd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.777194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c06eb88d-ceb6-4a3e-84d5-05eb1a97bd19 None None] MFA Rules not processed for user `56fb92e8e3f542c197d697cbce709cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:44.778388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c06eb88d-ceb6-4a3e-84d5-05eb1a97bd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:44.856631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 754/1502] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:44.869814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448745f-0a93-421c-876a-001b635db5aa None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.870057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448745f-0a93-421c-876a-001b635db5aa None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.870334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448745f-0a93-421c-876a-001b635db5aa None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.871928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 749/1503] 10.222.0.16 () {54 vars in 964 bytes} [Sat Sep 17 21:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:00:44.877489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2efeff-8a68-47a0-b009-fe2177c121ee None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:44.877793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2efeff-8a68-47a0-b009-fe2177c121ee None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:44.878064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2efeff-8a68-47a0-b009-fe2177c121ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:44.911078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:44.957339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-da2efeff-8a68-47a0-b009-fe2177c121ee None None] MFA Rules not processed for user `3f6f37bc49c64ed1a9a61accef657525`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:44.958593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-da2efeff-8a68-47a0-b009-fe2177c121ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.034644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 755/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Sat Sep 17 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:45.044252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448745f-0a93-421c-876a-001b635db5aa None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:45.045956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9448745f-0a93-421c-876a-001b635db5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.095397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:45.149115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448745f-0a93-421c-876a-001b635db5aa None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:45.222829 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:45.223433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.223700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.223964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.225570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.322232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:45.323655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:45.327454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:45.341809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 750/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:45.390043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03d90a-56a8-4314-ae62-1fe49db7e4b6 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.390371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03d90a-56a8-4314-ae62-1fe49db7e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.390671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c03d90a-56a8-4314-ae62-1fe49db7e4b6 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.392715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 756/1506] 10.222.0.16 () {52 vars in 830 bytes} [Sat Sep 17 21:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:00:45.395660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:45.396882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.493068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353cd446-ca71-41f3-996b-e3c53e28e821 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:45.583869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:45.584543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.584785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.585057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] PATH_INFO: `/v3/users/56fb92e8e3f542c197d697cbce709cae` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.597541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56fb92e8e3f542c197d697cbce709cae)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:45.598557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:45.598557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:00:45.598557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:00:45.599760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:45.609912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:45.665330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56fb92e8e3f542c197d697cbce709cae'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:45.674448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56fb92e8e3f542c197d697cbce709cae'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:45.674873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56fb92e8e3f542c197d697cbce709cae'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:45.689290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56fb92e8e3f542c197d697cbce709cae'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:45.712220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56fb92e8e3f542c197d697cbce709cae'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:45.717816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 751/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:00:45 2022] DELETE /identity/v3/users/56fb92e8e3f542c197d697cbce709cae => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:45.722175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:45.722632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:45.722779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:45.778705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:00:45.779863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.853619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:45.866153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 757/1508] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:00:45.876283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:45.877525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:45.977546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac19197-d1d0-4d97-b6b1-3269790ba1b9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:46.034670 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:46.055178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:46.055785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:46.056065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:46.056337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:46.058040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:46.164170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:46.165636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d667da-a19c-4102-ac15-3a929b52e226 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:46.184789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 752/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:00:46.538239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:00:46.539477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:00:46.551586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8549c30-d95c-4c99-a505-c509a763df8a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:00:46.650163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:00:46.650786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:00:46.651066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:00:46.651337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] PATH_INFO: `/v3/projects/d728b386858c43ed9fce0158ec5d53a1` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:00:46.662033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d728b386858c43ed9fce0158ec5d53a1)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:00:46.663058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:00:46.663058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:00:46.663058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:00:46.664259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:00:46.722707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd728b386858c43ed9fce0158ec5d53a1', 'request_id': 'req-ca475f7e-16f5-41ad-a883-f5916a200fea'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.728732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd728b386858c43ed9fce0158ec5d53a1', 'request_id': 'req-ca475f7e-16f5-41ad-a883-f5916a200fea'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.742393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd728b386858c43ed9fce0158ec5d53a1', 'request_id': 'req-ca475f7e-16f5-41ad-a883-f5916a200fea'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.767858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] The token cache is being invalidate because project d728b386858c43ed9fce0158ec5d53a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:00:46.768145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.768565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ca475f7e-16f5-41ad-a883-f5916a200fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:00:46.769951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 758/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:00:46 2022] DELETE /identity/v3/projects/d728b386858c43ed9fce0158ec5d53a1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:00:56.182459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=62 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:56.765896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:14.504653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c37ea6-1105-4bb2-a2d3-73e1422d01fc None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:14.505258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c37ea6-1105-4bb2-a2d3-73e1422d01fc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:14.505258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c37ea6-1105-4bb2-a2d3-73e1422d01fc None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:14.506588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 21:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:14.511372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76093a9f-5ada-41f2-83e3-c76e67957299 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:14.511820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76093a9f-5ada-41f2-83e3-c76e67957299 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:14.511926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76093a9f-5ada-41f2-83e3-c76e67957299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:14.593120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-76093a9f-5ada-41f2-83e3-c76e67957299 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:14.594337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-76093a9f-5ada-41f2-83e3-c76e67957299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:14.697956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 21:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:14.851773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c37ea6-1105-4bb2-a2d3-73e1422d01fc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:14.853910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c6c37ea6-1105-4bb2-a2d3-73e1422d01fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:14.973241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c37ea6-1105-4bb2-a2d3-73e1422d01fc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:14.978660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edc3c5-d52d-44dd-9135-8fd16ebe0872 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:14.979260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edc3c5-d52d-44dd-9135-8fd16ebe0872 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:14.979542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edc3c5-d52d-44dd-9135-8fd16ebe0872 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:14.979818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edc3c5-d52d-44dd-9135-8fd16ebe0872 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:14.988364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edc3c5-d52d-44dd-9135-8fd16ebe0872 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:14.990006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edc3c5-d52d-44dd-9135-8fd16ebe0872 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:15.001742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 754/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:16.088331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07762028-22f3-4f53-86f6-c31998cf7d85 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.088926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07762028-22f3-4f53-86f6-c31998cf7d85 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.088926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07762028-22f3-4f53-86f6-c31998cf7d85 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.090272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 21:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:16.097003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012b9d4-730d-498b-bc93-33287047f7ee None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.097458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012b9d4-730d-498b-bc93-33287047f7ee None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.097556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012b9d4-730d-498b-bc93-33287047f7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.117382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2012b9d4-730d-498b-bc93-33287047f7ee None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:16.118566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2012b9d4-730d-498b-bc93-33287047f7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:16.126493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:16.278892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07762028-22f3-4f53-86f6-c31998cf7d85 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:16.293651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07762028-22f3-4f53-86f6-c31998cf7d85 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:16.299931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7dbe7-0add-44eb-96ed-b7421890e376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:16.300533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7dbe7-0add-44eb-96ed-b7421890e376 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.300846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7dbe7-0add-44eb-96ed-b7421890e376 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.301153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7dbe7-0add-44eb-96ed-b7421890e376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.309660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7dbe7-0add-44eb-96ed-b7421890e376 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:16.311439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7dbe7-0add-44eb-96ed-b7421890e376 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:16.321121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 761/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:16.897392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29e6f8-fd07-4dd0-aa41-fc7317aff950 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.897677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29e6f8-fd07-4dd0-aa41-fc7317aff950 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.897951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29e6f8-fd07-4dd0-aa41-fc7317aff950 None None] PATH_INFO: `/v3` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.899227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 756/1517] 10.222.0.62 () {54 vars in 925 bytes} [Sat Sep 17 21:02:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:02:16.905953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5500a0-a763-4167-9c0e-8d72acca47dd None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:16.906238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5500a0-a763-4167-9c0e-8d72acca47dd None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:16.906513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5500a0-a763-4167-9c0e-8d72acca47dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:16.965300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9e5500a0-a763-4167-9c0e-8d72acca47dd None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:16.966445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9e5500a0-a763-4167-9c0e-8d72acca47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:16.982696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:17.036530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 762/1518] 10.222.0.62 () {58 vars in 1015 bytes} [Sat Sep 17 21:02:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:20.604940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd1c1-9b5b-49c8-b8c8-2a739fd82224 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.605374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd1c1-9b5b-49c8-b8c8-2a739fd82224 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.605477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd1c1-9b5b-49c8-b8c8-2a739fd82224 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.606792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 757/1519] 10.222.0.16 () {54 vars in 961 bytes} [Sat Sep 17 21:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:20.612255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ac1da-af28-466c-9409-899d324950fc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.612714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ac1da-af28-466c-9409-899d324950fc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.612828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ac1da-af28-466c-9409-899d324950fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.633888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5d1ac1da-af28-466c-9409-899d324950fc None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:20.635079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5d1ac1da-af28-466c-9409-899d324950fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:20.643791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 763/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 17 21:02:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:20.651307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd1c1-9b5b-49c8-b8c8-2a739fd82224 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:20.665063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd1c1-9b5b-49c8-b8c8-2a739fd82224 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:20.670754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65ad88-24bd-4b4f-8c6e-4d93d346601c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:20.671345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65ad88-24bd-4b4f-8c6e-4d93d346601c service placement] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:20.671617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65ad88-24bd-4b4f-8c6e-4d93d346601c service placement] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:20.671888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65ad88-24bd-4b4f-8c6e-4d93d346601c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:20.680138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65ad88-24bd-4b4f-8c6e-4d93d346601c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:20.681771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65ad88-24bd-4b4f-8c6e-4d93d346601c service placement] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:20.691321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 758/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:27.799948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8268-427c-4f89-bfdc-b151ff186556 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.800563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8268-427c-4f89-bfdc-b151ff186556 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.800563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8268-427c-4f89-bfdc-b151ff186556 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.801875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 21:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:27.807728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8bce-0455-4927-a057-215ccac18d13 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:27.808135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8bce-0455-4927-a057-215ccac18d13 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:27.808272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8bce-0455-4927-a057-215ccac18d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:27.828558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-913a8bce-0455-4927-a057-215ccac18d13 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:27.829780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-913a8bce-0455-4927-a057-215ccac18d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:27.837553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:27.992047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c8268-427c-4f89-bfdc-b151ff186556 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:28.007410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c8268-427c-4f89-bfdc-b151ff186556 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:28.013095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1dd44-4310-49e9-87d8-ec41eee09f3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:28.013681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1dd44-4310-49e9-87d8-ec41eee09f3a service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:28.013950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1dd44-4310-49e9-87d8-ec41eee09f3a service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:28.014218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1dd44-4310-49e9-87d8-ec41eee09f3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:28.022279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1dd44-4310-49e9-87d8-ec41eee09f3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:28.023844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1dd44-4310-49e9-87d8-ec41eee09f3a service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:28.033542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 765/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:29.114341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c453679-c201-4a12-afe0-41119eff8479 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.114902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c453679-c201-4a12-afe0-41119eff8479 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.114902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c453679-c201-4a12-afe0-41119eff8479 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.116154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 17 21:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:29.122221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c813c5-8d35-496c-8c0f-e6b2980c553b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.123206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c813c5-8d35-496c-8c0f-e6b2980c553b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.123343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c813c5-8d35-496c-8c0f-e6b2980c553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.143064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-32c813c5-8d35-496c-8c0f-e6b2980c553b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:29.144255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-32c813c5-8d35-496c-8c0f-e6b2980c553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:29.151939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 17 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:29.303798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c453679-c201-4a12-afe0-41119eff8479 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:29.318437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c453679-c201-4a12-afe0-41119eff8479 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:29.323990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a2b9a-9e44-49be-ac61-93468156e076 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:29.324570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a2b9a-9e44-49be-ac61-93468156e076 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:29.324873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a2b9a-9e44-49be-ac61-93468156e076 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:29.325143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a2b9a-9e44-49be-ac61-93468156e076 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:29.333218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a2b9a-9e44-49be-ac61-93468156e076 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:29.334776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a2b9a-9e44-49be-ac61-93468156e076 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:29.343882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 761/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:33.559295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08528c-1cab-4938-8cb8-a7cbcf5b7292 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.560313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08528c-1cab-4938-8cb8-a7cbcf5b7292 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.560437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08528c-1cab-4938-8cb8-a7cbcf5b7292 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.561812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 21:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:33.567031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535bf4-f872-4bb7-b075-28603161da7c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.567486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535bf4-f872-4bb7-b075-28603161da7c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.567591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535bf4-f872-4bb7-b075-28603161da7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.588369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-92535bf4-f872-4bb7-b075-28603161da7c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:33.589581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-92535bf4-f872-4bb7-b075-28603161da7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:33.597310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 21:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:33.886455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08528c-1cab-4938-8cb8-a7cbcf5b7292 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:33.888269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-be08528c-1cab-4938-8cb8-a7cbcf5b7292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:33.964520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08528c-1cab-4938-8cb8-a7cbcf5b7292 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:33.968272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f08f15-7e0c-4940-b3c1-a86fe47f684c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:33.968897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08f15-7e0c-4940-b3c1-a86fe47f684c service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:33.969175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08f15-7e0c-4940-b3c1-a86fe47f684c service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:33.969448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f08f15-7e0c-4940-b3c1-a86fe47f684c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:33.978198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f08f15-7e0c-4940-b3c1-a86fe47f684c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:33.979639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f08f15-7e0c-4940-b3c1-a86fe47f684c service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:33.989363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 768/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:36.137769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80567844-d097-46f5-8d23-d9b2245dc4bb None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:36.139000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80567844-d097-46f5-8d23-d9b2245dc4bb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:36.139000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80567844-d097-46f5-8d23-d9b2245dc4bb None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:36.140147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:36.145382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29da05-87a6-4c95-9627-1a7c3ba45216 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:36.145826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29da05-87a6-4c95-9627-1a7c3ba45216 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:36.145946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29da05-87a6-4c95-9627-1a7c3ba45216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:36.166195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9e29da05-87a6-4c95-9627-1a7c3ba45216 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:36.167390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9e29da05-87a6-4c95-9627-1a7c3ba45216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:36.175294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 21:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:36.468923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80567844-d097-46f5-8d23-d9b2245dc4bb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:36.474284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=75 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:36.481058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80567844-d097-46f5-8d23-d9b2245dc4bb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:36.484932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddb95a-0405-49f4-8128-8981c52a88b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:36.485525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddb95a-0405-49f4-8128-8981c52a88b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:36.485801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddb95a-0405-49f4-8128-8981c52a88b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:36.486068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddb95a-0405-49f4-8128-8981c52a88b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:36.494824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddb95a-0405-49f4-8128-8981c52a88b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:36.496272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddb95a-0405-49f4-8128-8981c52a88b6 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:36.506283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 764/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:40.213911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54ab92-2668-497a-a7ff-595755ee9dc1 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:40.214491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54ab92-2668-497a-a7ff-595755ee9dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:40.214491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54ab92-2668-497a-a7ff-595755ee9dc1 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:40.216344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 21:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:40.221443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2084fd-9c68-4450-bd32-56ff742af06e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:40.221878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2084fd-9c68-4450-bd32-56ff742af06e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:40.222023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2084fd-9c68-4450-bd32-56ff742af06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:40.241974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0a2084fd-9c68-4450-bd32-56ff742af06e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:40.243198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0a2084fd-9c68-4450-bd32-56ff742af06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:40.251295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 21:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:40.550926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54ab92-2668-497a-a7ff-595755ee9dc1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:40.552549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9a54ab92-2668-497a-a7ff-595755ee9dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:40.615907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:40.629618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54ab92-2668-497a-a7ff-595755ee9dc1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:40.633198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b5e253-56f0-4350-8f20-0f054045f9b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:40.633793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e253-56f0-4350-8f20-0f054045f9b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:40.634071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e253-56f0-4350-8f20-0f054045f9b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:40.634347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5e253-56f0-4350-8f20-0f054045f9b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:40.643980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5e253-56f0-4350-8f20-0f054045f9b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:40.645421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b5e253-56f0-4350-8f20-0f054045f9b4 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:40.655271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 771/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:42.825814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000ed37-b4a8-402e-9e19-5709ff2a4331 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:42.826339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000ed37-b4a8-402e-9e19-5709ff2a4331 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:42.826339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000ed37-b4a8-402e-9e19-5709ff2a4331 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:42.827701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 17 21:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:02:42.832874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f459eeb-71eb-4545-9963-bc711c19e315 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:42.833272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f459eeb-71eb-4545-9963-bc711c19e315 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:42.833418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f459eeb-71eb-4545-9963-bc711c19e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:42.852448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0f459eeb-71eb-4545-9963-bc711c19e315 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:02:42.853779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0f459eeb-71eb-4545-9963-bc711c19e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:42.862058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 17 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:02:43.165922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000ed37-b4a8-402e-9e19-5709ff2a4331 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:02:43.167894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0000ed37-b4a8-402e-9e19-5709ff2a4331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:02:43.182449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000ed37-b4a8-402e-9e19-5709ff2a4331 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:02:43.186176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0242b-9c75-4c8e-b1b2-d6ee5980887e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:02:43.186767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0242b-9c75-4c8e-b1b2-d6ee5980887e service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:02:43.187047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0242b-9c75-4c8e-b1b2-d6ee5980887e service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:02:43.187319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0242b-9c75-4c8e-b1b2-d6ee5980887e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:02:43.196012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0242b-9c75-4c8e-b1b2-d6ee5980887e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:02:43.197461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0242b-9c75-4c8e-b1b2-d6ee5980887e service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:02:43.207020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 767/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:02:52.847627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=11 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:53.202858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=14 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:07.804237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.805258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.805699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.841847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:07.843583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:07.858736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 773/1540] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:07.867785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707a328-6b2b-491e-a371-44a2078254bd None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.868773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707a328-6b2b-491e-a371-44a2078254bd None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.869229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707a328-6b2b-491e-a371-44a2078254bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.895714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0707a328-6b2b-491e-a371-44a2078254bd None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:07.897255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0707a328-6b2b-491e-a371-44a2078254bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:07.908586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 768/1541] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:07.913127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:07.923864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba351d0-7c17-470d-be78-0eee6172647c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:07.939204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:07.940152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.940684 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.941209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.942071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:07.942590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:07.944557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:07.959042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d241a-c50f-49e6-bc21-f408c3d9a9e4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.959669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d241a-c50f-49e6-bc21-f408c3d9a9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.960189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d241a-c50f-49e6-bc21-f408c3d9a9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.960809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 774/1542] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:07.970023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:07.980429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76d16b-4a4a-4d83-9fa6-1efc6e146091 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:07.991913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:07.992808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:07.993120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-246d241a-c50f-49e6-bc21-f408c3d9a9e4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:07.993253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:07.993519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:07.994171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:07.995768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:04:07.995768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:04:07.995768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:04:07.996204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-246d241a-c50f-49e6-bc21-f408c3d9a9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:07.997316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.015277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 769/1543] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:08.024404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.025149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.025669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.056344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.064700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.078783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 770/1544] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:08.084054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.085249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 775/1545] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:07 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:08.089979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.102106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3bd1e-70ee-4f99-bf1a-e5dc032c6e14 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.216283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.217375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.217925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.218424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.219335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:08.219870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.221170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:04:08.221170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:04:08.221170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:04:08.222498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.235492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9991d3-67e3-48be-90f7-0cb9dfee2af0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.246418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 771/1546] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:04:08.252994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.252994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.252994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.252994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.253370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.254915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.256039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.270018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431d836-58b9-4a99-8406-fb5a765fbc8e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.282877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0532946-f490-4576-85fb-81181691039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.283957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0532946-f490-4576-85fb-81181691039d admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.284454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0532946-f490-4576-85fb-81181691039d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.285022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0532946-f490-4576-85fb-81181691039d admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.285921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0532946-f490-4576-85fb-81181691039d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.288388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0532946-f490-4576-85fb-81181691039d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.296449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 776/1547] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:08.303895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.319386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2e7f1-3592-46c9-aaeb-816492716efd admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.334056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.335175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.340979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.341654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.343579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.355267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.359194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 772/1548] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:08.361566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.362886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.366399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.379625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 777/1549] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:08.387374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.413707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f8648-dd14-4750-9116-7df8feed6da8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.425757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.427112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.515289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bfe6d-cdd3-4c31-90b6-f5c62f3dda65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.516517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bfe6d-cdd3-4c31-90b6-f5c62f3dda65 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.517006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bfe6d-cdd3-4c31-90b6-f5c62f3dda65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.517568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bfe6d-cdd3-4c31-90b6-f5c62f3dda65 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.518566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bfe6d-cdd3-4c31-90b6-f5c62f3dda65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.522313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bfe6d-cdd3-4c31-90b6-f5c62f3dda65 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.550041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 773/1550] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.551303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.563938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 778/1551] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/users => generated 302 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:08.565360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140049f4-2f8c-4aad-8879-e29fcc22d7e0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.565799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b87aa-c66a-46e2-b4aa-409393eed1e4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.566275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b87aa-c66a-46e2-b4aa-409393eed1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.566844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b87aa-c66a-46e2-b4aa-409393eed1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.570879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da21fd7-e521-44b2-87ae-4ee10e51130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.571479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21fd7-e521-44b2-87ae-4ee10e51130e admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.571752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21fd7-e521-44b2-87ae-4ee10e51130e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.572077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da21fd7-e521-44b2-87ae-4ee10e51130e admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.573005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da21fd7-e521-44b2-87ae-4ee10e51130e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.575241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da21fd7-e521-44b2-87ae-4ee10e51130e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.587849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 774/1552] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:08.590142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d96d-6add-49e0-a7bc-f2fb85157c1a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.590585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d96d-6add-49e0-a7bc-f2fb85157c1a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.590942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425d96d-6add-49e0-a7bc-f2fb85157c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.591618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-855b87aa-c66a-46e2-b4aa-409393eed1e4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.592979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-855b87aa-c66a-46e2-b4aa-409393eed1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.601769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 779/1553] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.602993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.603261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.603535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.614923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b425d96d-6add-49e0-a7bc-f2fb85157c1a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.616114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b425d96d-6add-49e0-a7bc-f2fb85157c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.624773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 775/1554] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.624945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.626031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0eb02-49b1-44e0-9868-4bce1cea1951 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.626154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.626327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0eb02-49b1-44e0-9868-4bce1cea1951 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.626579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0eb02-49b1-44e0-9868-4bce1cea1951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.634111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 780/1555] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.634897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.642047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:08.645862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772b328f-5b44-4c71-95a8-eaf8cfb9932f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.647095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-10e0eb02-49b1-44e0-9868-4bce1cea1951 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.648368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-10e0eb02-49b1-44e0-9868-4bce1cea1951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.652508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e0f47-9c57-4a2d-b8fa-44f58d75fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.653618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0f47-9c57-4a2d-b8fa-44f58d75fff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.654321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0f47-9c57-4a2d-b8fa-44f58d75fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.654634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0f47-9c57-4a2d-b8fa-44f58d75fff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.655293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e0f47-9c57-4a2d-b8fa-44f58d75fff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.656938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e0f47-9c57-4a2d-b8fa-44f58d75fff5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.657336 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 776/1556] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.661291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.661291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.661291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.669196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 781/1557] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:08.670331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630758e-ccb0-4484-be32-8ed524dbe63c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.670618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630758e-ccb0-4484-be32-8ed524dbe63c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.670894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630758e-ccb0-4484-be32-8ed524dbe63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.680033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.681446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.690447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 777/1558] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.691240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.700852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e630758e-ccb0-4484-be32-8ed524dbe63c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.701850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92879187-079a-47e8-8682-438eaa8ad92a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.702056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e630758e-ccb0-4484-be32-8ed524dbe63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.706478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0c821-c6e3-4e4f-9b78-055b1cef53d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.707126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0c821-c6e3-4e4f-9b78-055b1cef53d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.707411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0c821-c6e3-4e4f-9b78-055b1cef53d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.707687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0c821-c6e3-4e4f-9b78-055b1cef53d7 admin admin] PATH_INFO: `/v3/projects/3594924fe4fe453aa9a1c8f4c860779d/users/f7412e3b2d124a5994fc9ddb726b9865/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.710162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 782/1559] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.711381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4698-2f04-4916-b598-fdbd0f3831c9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.711641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4698-2f04-4916-b598-fdbd0f3831c9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.711917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b4698-2f04-4916-b598-fdbd0f3831c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.724389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0c821-c6e3-4e4f-9b78-055b1cef53d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3594924fe4fe453aa9a1c8f4c860779d, user_id=f7412e3b2d124a5994fc9ddb726b9865, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.727108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0c821-c6e3-4e4f-9b78-055b1cef53d7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.734966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2a4b4698-2f04-4916-b598-fdbd0f3831c9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.736192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2a4b4698-2f04-4916-b598-fdbd0f3831c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.736594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:08.743265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 778/1560] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:08 2022] PUT /identity/v3/projects/3594924fe4fe453aa9a1c8f4c860779d/users/f7412e3b2d124a5994fc9ddb726b9865/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:08.744647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.745573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.746087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.766717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.767903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.799176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 783/1561] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.803307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.803307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.803307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.826567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.827680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.835690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 779/1562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.836164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 784/1563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.837034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.838198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.849630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea2041-5bd2-48be-9ba7-7ee23d5711f7 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.850000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f66cf-c3c6-4b97-b88b-9d5698cf422c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.854175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743753e8-33db-4761-ab1e-97a6773f475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.855071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.855393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743753e8-33db-4761-ab1e-97a6773f475e admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.855531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743753e8-33db-4761-ab1e-97a6773f475e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.855688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.855846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743753e8-33db-4761-ab1e-97a6773f475e admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.856117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.856480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743753e8-33db-4761-ab1e-97a6773f475e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.856577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.857206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:08.857502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.858145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743753e8-33db-4761-ab1e-97a6773f475e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.859078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0526c8e-c055-491b-9558-d245b574b655 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.868444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 785/1564] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:08.869728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947d8c8-bb21-4107-8014-49f934c9f4db None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.870008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947d8c8-bb21-4107-8014-49f934c9f4db None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.870990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947d8c8-bb21-4107-8014-49f934c9f4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.878674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 780/1565] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:08.878674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.878674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.878674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.890791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b947d8c8-bb21-4107-8014-49f934c9f4db None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.891994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b947d8c8-bb21-4107-8014-49f934c9f4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.892598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.893808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.900091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 786/1566] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.901312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.901585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.901759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 781/1567] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:08.901848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.902531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.911706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd6fc5e-ec86-4d41-a509-fa408ed31762 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.916325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.917427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.917725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.918002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.918639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:08.918936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.920531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.932066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] MFA Rules not processed for user `f7412e3b2d124a5994fc9ddb726b9865`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:08.932425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 782/1568] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:08.933798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.933943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:08.949612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7269b2-50f0-4ec3-ba0b-41e0e9c5d4ab admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.954191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.954807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.955087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.955359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.956000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:08.956303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.957913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:08.969745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 783/1569] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:08.970648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:08.980472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbeb8e-d47a-4673-8b1a-c729b2f87289 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:08.985042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:08.985621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:08.985903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:08.986289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:08.986929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:08.987230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:08.988838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.000753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 784/1570] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:09.001529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.005685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 787/1571] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:09.006573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.011622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d92b9f-dc6b-44d0-80e8-ac0865ecfcd5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.016201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.016891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.017178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.017304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f9c80-a4d9-4245-97a5-9bd147269cc4 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.017453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] PATH_INFO: `/v3/projects/167bcd812954490e99cc625840fb5dd2/users/577fbc32bd544d228165ab9bedc5a14c/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.021665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.022241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.022512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.022777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.023390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.025323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.032159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] RBAC: Authorizing `identity:create_grant(project_id=167bcd812954490e99cc625840fb5dd2, user_id=577fbc32bd544d228165ab9bedc5a14c, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.034848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.049073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 785/1572] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:09 2022] PUT /identity/v3/projects/167bcd812954490e99cc625840fb5dd2/users/577fbc32bd544d228165ab9bedc5a14c/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:09.049897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.052968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 788/1573] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.053781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.087572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:09.115683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ef2b5-81a9-465a-9556-eb1014ca804b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.118494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe456a5-bb68-4cdc-b2e3-98520aa7e4f6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.121126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.121727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.122008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.122284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.122928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:09.123230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.124140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.124790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.124896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.125067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.125342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.125985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:09.126284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.127885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.136906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 786/1574] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:09.137766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.147088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 789/1575] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:09.147088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.147873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1bb5-7174-472b-bdf3-3c5627c31688 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.152356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.153436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.153436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.153650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.154123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.156013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.156754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ad5c3-6eb0-40bc-9128-9debfb09b723 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.161353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:09.162048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.162646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.162988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.163196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.163844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.165828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.186802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 787/1576] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.188033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.198155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 790/1577] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.199031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.252640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb5f9d-f4f9-4ed1-b090-9516b7fe492a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.257322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fb34b-21ac-46be-a65d-fed30d934dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.257929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb34b-21ac-46be-a65d-fed30d934dfe admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.258212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb34b-21ac-46be-a65d-fed30d934dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.258490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fb34b-21ac-46be-a65d-fed30d934dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.259116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fb34b-21ac-46be-a65d-fed30d934dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.261092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fb34b-21ac-46be-a65d-fed30d934dfe admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.261458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e9f93-7b19-4fa3-9612-0d83d5639714 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.265986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.266577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.266850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.267128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.267744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.269676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.288666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 788/1578] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.290022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.290642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.290979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.298219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 791/1579] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.298895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.310891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:09.312075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:09.359965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27743561-e215-4e8b-a9a0-67c9e3fa70f8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.364557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.365181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.365453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.365722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.366363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.368274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.369543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 789/1580] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:09.371491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.384170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92a01b-ba11-4535-a65d-25040b69e4c4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.390817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.391429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.391711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.391989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.392620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.394775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.397356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 792/1581] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.398165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.410309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97993913-11a4-4e5f-a06a-f45b6f849612 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.414823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.415407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.415677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.416022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.416725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.419005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.426973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 790/1582] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.427965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.436972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:09.451698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 793/1583] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.452656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.481979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:09.496205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaee9fb-3115-4cc5-a875-794ef3afda27 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.520762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ceceba-7e2e-474d-9ed5-61af104a8028 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.525363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.525964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.526247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.526548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.527209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.529154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.550861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 794/1584] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.551762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.556078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.556842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.557116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.557387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.558720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.559727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:04:09.559727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:04:09.559727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:04:09.560950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.562358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc922fa9-f237-417e-abdb-e8117822ad19 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.566889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.567468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.567741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.568011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.568655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.570580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.584566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 791/1585] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.585454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.596561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06042b4f-2fa3-4335-bff5-bbb34d87f772 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.597023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 795/1586] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.600892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.601328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.601922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.602202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.602480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.603105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.606106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.628575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 792/1587] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.629198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.638848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1e026-bbbe-4ff7-87b8-41950145faa9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.643841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.644426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.644737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.645016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.645646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.647243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.657742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c6673-1780-426e-acce-de3f139b73b1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.658456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 793/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.658990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.663234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.663838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.664115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.664388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.669232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad2847-b55e-415e-b15b-a001517ded63 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.674286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.674595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.675181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.675434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.675703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.675829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.676309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.678233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.685188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 796/1589] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:09.686033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.695987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260d08f-5f38-4528-b7d7-63dbb80bbdb8 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.701712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.702368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.702582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 794/1590] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.702718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.702899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.703351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.703595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.705770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.718242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcba79-2a65-46f4-a917-d48a978008e1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.724972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.725813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.726090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.726367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.727246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.728863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.735292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 797/1591] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:09 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:09.735292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.743218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 795/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.743218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.744240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4896a-b683-4eb9-b65a-4bf72196fecc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.750150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.750767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.751044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.751323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.751506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00213bc3-9243-49b5-9566-2fd7614534d1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.751967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.753745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.756485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.757239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.757465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.757738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.758551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.760182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.766558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 798/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.767164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.776373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 796/1594] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.778062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.779028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790d362-1960-4d4d-85e0-3e2dce263141 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.783566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.784175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.784456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.784760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.785403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.787020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.790547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12096e84-c979-4141-815b-2240616b5c2c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.798075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.798673 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.798994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.799274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.800393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 799/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.800509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.801430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.802043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.813258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e9166-d817-4ccc-90a1-26e4ffe8d062 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.814381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 797/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.818580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.818797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.819347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.819626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.819983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.820724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.822652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.831111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14bde-ce9e-47b3-8b95-ec39b59d4d03 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.836937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 800/1597] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.836937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.839720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.840334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.840616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.840919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.841593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.843241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.848110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7901e-3682-4426-b2df-2023375c6c40 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.854014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.854937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.855208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.855479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.856209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.856653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 798/1598] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.857558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.857960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.867727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f676b8-1906-48a3-833c-30ec5651dd64 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.869033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 801/1599] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.869940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.876241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.876885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.877167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.877447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.878294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.879751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.880329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac026e-f565-4c67-827c-9efdfe8cef73 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.888121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.889319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.889599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.890033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] PATH_INFO: `/v3/projects/7ed6be4e5067451e983fc83572330989/users/913661574f034136b64d3a05ef7c6c64/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.894690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 799/1600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.894690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.895035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:09.903937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c4fc2-2f7a-45ec-b158-80fe6dc8e2e6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.909068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:09.909760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:09.910179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:09.910462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:09.911383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.911532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed6be4e5067451e983fc83572330989, user_id=913661574f034136b64d3a05ef7c6c64, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:09.912755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:04:09.912755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:04:09.912755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:04:09.913421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.917820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:09.930816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 802/1601] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:09 2022] PUT /identity/v3/projects/7ed6be4e5067451e983fc83572330989/users/913661574f034136b64d3a05ef7c6c64/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:09.931828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.938554 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 800/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:09.940098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:09.950625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0e87b-9d73-40cd-9fbc-f815730562f4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:09.987026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:10.020358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.021364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.021646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.021918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.022613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.023300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cc5d8-16d7-4314-854a-25c132146686 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.024382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.037245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 801/1603] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:10.038474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.045538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.046228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.046508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.046779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] PATH_INFO: `/v3/projects/d9689fd1a04647cf91a73680e4b677d1/users/1e3b1e0f4888400b89c47b3e44443228/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.052047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8ec3e-5616-4a06-b3b1-a93bb4746eed admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.059533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.060730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.061584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.062284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.063166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.064093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9689fd1a04647cf91a73680e4b677d1, user_id=1e3b1e0f4888400b89c47b3e44443228, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.065016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.067636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.077395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 802/1604] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:10.078397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.083291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 803/1605] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:09 2022] PUT /identity/v3/projects/d9689fd1a04647cf91a73680e4b677d1/users/1e3b1e0f4888400b89c47b3e44443228/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.084128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.089944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614e484-f8bd-4a4a-86d8-27cf47cb4725 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.163986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51046bfe-869a-4196-8e38-25544bd3f3c3 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.169932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.170320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.170640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.170909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] PATH_INFO: `/v3/projects/1e42842c724f480f9cdd5ea75707b43f/users/a3dac44116784a9e963a69e4793bac4b/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.181889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.183789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.184233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.184960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] PATH_INFO: `/v3/projects/9cdcf03c8c5e456b8f8ea08564ad6c94/users/dc3465a68b04477193d6c88130dab151/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.188935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e42842c724f480f9cdd5ea75707b43f, user_id=a3dac44116784a9e963a69e4793bac4b, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.192291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.208320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdcf03c8c5e456b8f8ea08564ad6c94, user_id=dc3465a68b04477193d6c88130dab151, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.209518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 804/1606] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:10 2022] PUT /identity/v3/projects/1e42842c724f480f9cdd5ea75707b43f/users/a3dac44116784a9e963a69e4793bac4b/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.210409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.212093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.232293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 803/1607] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:10 2022] PUT /identity/v3/projects/9cdcf03c8c5e456b8f8ea08564ad6c94/users/dc3465a68b04477193d6c88130dab151/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.234974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.286058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:10.299226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ae65-b7ea-4c0f-a98e-5742e15d054a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.317913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22296488-0d67-4703-a225-fdefb6066710 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.323867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.324648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.325093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.325452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] PATH_INFO: `/v3/projects/1265ca6c66b24a0e9a98331fb134c262/users/4784fbffe44e463f9cb71c0195ced4d9/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.341699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1265ca6c66b24a0e9a98331fb134c262, user_id=4784fbffe44e463f9cb71c0195ced4d9, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.344844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.360332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 804/1608] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:10 2022] PUT /identity/v3/projects/1265ca6c66b24a0e9a98331fb134c262/users/4784fbffe44e463f9cb71c0195ced4d9/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.361599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.367731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.368416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.368763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.369016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.369776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.371490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.382814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 805/1609] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:10.383693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.394786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec27744-cae0-4a5d-a026-8dfadb33a504 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.415325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:10.465315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550bde4-3efc-417c-9a8f-f5bdde127653 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.469350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7f3e2-61a7-4b32-9fa2-8565865628da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.470119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dce060-7ce9-4cf1-bcef-4430a83012f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.470253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f3e2-61a7-4b32-9fa2-8565865628da admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.470499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f3e2-61a7-4b32-9fa2-8565865628da admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.470720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f3e2-61a7-4b32-9fa2-8565865628da admin admin] PATH_INFO: `/v3/projects/aae15e7cf5d74f9f90e067bb801c7e95/users/4b5e98fb43a143ea8cfa30990b4fffc2/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.470858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dce060-7ce9-4cf1-bcef-4430a83012f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.471109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dce060-7ce9-4cf1-bcef-4430a83012f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.471385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dce060-7ce9-4cf1-bcef-4430a83012f5 admin admin] PATH_INFO: `/v3/projects/43d4440dd2464bcb95e25ead37c5658a/users/1fa4420540af41e4967b0f731f818af5/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.488929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7f3e2-61a7-4b32-9fa2-8565865628da admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae15e7cf5d74f9f90e067bb801c7e95, user_id=4b5e98fb43a143ea8cfa30990b4fffc2, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.490241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dce060-7ce9-4cf1-bcef-4430a83012f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d4440dd2464bcb95e25ead37c5658a, user_id=1fa4420540af41e4967b0f731f818af5, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.491628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7f3e2-61a7-4b32-9fa2-8565865628da admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.494169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dce060-7ce9-4cf1-bcef-4430a83012f5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.507317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 806/1610] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:10 2022] PUT /identity/v3/projects/aae15e7cf5d74f9f90e067bb801c7e95/users/4b5e98fb43a143ea8cfa30990b4fffc2/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.509250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b405-13cb-4aa1-8651-a234e28bbfe7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.509807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b405-13cb-4aa1-8651-a234e28bbfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.510223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54b405-13cb-4aa1-8651-a234e28bbfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.514818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 805/1611] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:10 2022] PUT /identity/v3/projects/43d4440dd2464bcb95e25ead37c5658a/users/1fa4420540af41e4967b0f731f818af5/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.519828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3c83-e61f-4dd7-b0f6-49450a93f1bb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.520125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3c83-e61f-4dd7-b0f6-49450a93f1bb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.520461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd3c83-e61f-4dd7-b0f6-49450a93f1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.541526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1c54b405-13cb-4aa1-8651-a234e28bbfe7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.541526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1c54b405-13cb-4aa1-8651-a234e28bbfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.543832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-dccd3c83-e61f-4dd7-b0f6-49450a93f1bb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.545180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-dccd3c83-e61f-4dd7-b0f6-49450a93f1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.607397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 806/1612] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.610479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.610479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.610479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.618578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 807/1613] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.620037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72515599-36f2-4e53-8751-fad4d8c7cb72 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.621014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72515599-36f2-4e53-8751-fad4d8c7cb72 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.621361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72515599-36f2-4e53-8751-fad4d8c7cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.648196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] MFA Rules not processed for user `dc3465a68b04477193d6c88130dab151`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.649433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.650917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-72515599-36f2-4e53-8751-fad4d8c7cb72 None None] MFA Rules not processed for user `4784fbffe44e463f9cb71c0195ced4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.652418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-72515599-36f2-4e53-8751-fad4d8c7cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.657948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:10.722868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 807/1614] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.723704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.734240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 808/1615] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.734491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb50f5f-7f87-486e-affb-6981f4605699 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.735290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a896f-17aa-4cd2-8b1d-06c1bf706c88 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.735567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a896f-17aa-4cd2-8b1d-06c1bf706c88 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.735832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a896f-17aa-4cd2-8b1d-06c1bf706c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.739360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af0ea5-c917-47d7-ac48-84397d53fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.740208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0ea5-c917-47d7-ac48-84397d53fedf admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.740505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0ea5-c917-47d7-ac48-84397d53fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.740752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af0ea5-c917-47d7-ac48-84397d53fedf admin admin] PATH_INFO: `/v3/projects/7ed6be4e5067451e983fc83572330989/users/913661574f034136b64d3a05ef7c6c64/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.743407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af0ea5-c917-47d7-ac48-84397d53fedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed6be4e5067451e983fc83572330989, user_id=913661574f034136b64d3a05ef7c6c64, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.747172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af0ea5-c917-47d7-ac48-84397d53fedf admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.767882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 808/1616] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:10 2022] PUT /identity/v3/projects/7ed6be4e5067451e983fc83572330989/users/913661574f034136b64d3a05ef7c6c64/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:10.767882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.767882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.767882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.768383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c26a896f-17aa-4cd2-8b1d-06c1bf706c88 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.768383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c26a896f-17aa-4cd2-8b1d-06c1bf706c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.785690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.786895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.806971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:10.829203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 809/1617] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.830553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.830837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.831114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.851970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 809/1618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.853008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.854268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.862679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] MFA Rules not processed for user `577fbc32bd544d228165ab9bedc5a14c`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:10.863878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:10.911957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f8f0b-503f-4ee1-8ab3-682277a92ed5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.915487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.916083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.916359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.916627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.925572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.926989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.927879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 810/1619] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:10.931805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.938321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 810/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:10.938853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d4865-25d5-4af3-ad65-71ebd22a5d85 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:10.939151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:10.942407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab011f-255b-4869-a484-7902f51ed64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:10.942994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab011f-255b-4869-a484-7902f51ed64c service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.943270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab011f-255b-4869-a484-7902f51ed64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.943540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab011f-255b-4869-a484-7902f51ed64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.954411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab011f-255b-4869-a484-7902f51ed64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:10.955854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab011f-255b-4869-a484-7902f51ed64c service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:10.966099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 811/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:10.967488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6b88c-e9be-4ac3-9820-1afa474940ab None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.967633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6b88c-e9be-4ac3-9820-1afa474940ab None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.967872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6b88c-e9be-4ac3-9820-1afa474940ab None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:10.973615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 812/1622] 10.222.0.16 () {54 vars in 965 bytes} [Sat Sep 17 21:04:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:10.975031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010d16-a842-4166-bae5-20c23085346d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:10.975316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010d16-a842-4166-bae5-20c23085346d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:10.975591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49010d16-a842-4166-bae5-20c23085346d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.009322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec43238-1f0c-447f-8bde-a7ffa98e2fa7 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.015459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.016521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.016877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.017296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.020122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-49010d16-a842-4166-bae5-20c23085346d None None] MFA Rules not processed for user `913661574f034136b64d3a05ef7c6c64`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:11.022176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-49010d16-a842-4166-bae5-20c23085346d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:11.042588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:11.103219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.105118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.122886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 811/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:11.125622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.143873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87cdd5-d169-46ca-b430-7b57e5301410 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.149860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.150813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.151213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.151811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.170733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 813/1624] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:11.171921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010d16-a842-4166-bae5-20c23085346d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.183121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.186887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.203939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 812/1625] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:11.205306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.207131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49010d16-a842-4166-bae5-20c23085346d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.215983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e3938-238a-4256-bc0b-1bd4722b69f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.217640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e3938-238a-4256-bc0b-1bd4722b69f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.217923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e3938-238a-4256-bc0b-1bd4722b69f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.218195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e3938-238a-4256-bc0b-1bd4722b69f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.219348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f940421-9310-468b-a635-b9e6f45ff65a service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.237074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.238532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e3938-238a-4256-bc0b-1bd4722b69f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.238960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.239257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.239540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.240195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.240538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e3938-238a-4256-bc0b-1bd4722b69f6 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.247515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.255361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 814/1626] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:11.256787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.257174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.257547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.276894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:11.310296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 813/1627] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:11 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:11.314470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.352514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:11.355931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:11.473734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570e3ae-377b-4dac-9cba-dd02821a0d33 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.482903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.484329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.484992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.485488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.493671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 815/1628] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 17 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:11.496372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.587182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f9134-e087-45a6-98e4-ffa616b5cf70 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.594640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.595253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.597012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.597856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.598024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.600198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.624433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.625988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.634896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 816/1629] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:11 2022] POST /identity/v3/users => generated 309 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:11.636763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.644260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 814/1630] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:11.658172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.658975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:11.667837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a41e-4830-4a81-9e16-d5fc37f23dde admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.684591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.685641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.686417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525031ed-8bcc-4c96-9871-37570de7c074 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.687078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.688034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.697598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.699733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.701294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.701594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.703557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.708030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.743230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 815/1631] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:11.746717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.763594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fed91-c57a-45ed-97a1-60e7ce18ec9b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.769428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.770738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.771068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.771613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] PATH_INFO: `/v3/projects/4153e5736f014d42a74ed38f14028dd4/users/a5678b3f40f146c4aeee6ecaefb2f37c/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.773163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.776000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.793713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 817/1632] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:11.809547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4153e5736f014d42a74ed38f14028dd4, user_id=a5678b3f40f146c4aeee6ecaefb2f37c, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:11.810997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:11.828547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 816/1633] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:11 2022] PUT /identity/v3/projects/4153e5736f014d42a74ed38f14028dd4/users/a5678b3f40f146c4aeee6ecaefb2f37c/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:11.833558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:11.860099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a870fc1-ec7b-4284-a820-976353c7b3b5 service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:11.992069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18852536-4a24-4c06-8e7e-3d8844987dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:11.992893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18852536-4a24-4c06-8e7e-3d8844987dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:11.993184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18852536-4a24-4c06-8e7e-3d8844987dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:11.993526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18852536-4a24-4c06-8e7e-3d8844987dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:11.994375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18852536-4a24-4c06-8e7e-3d8844987dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:12.004276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18852536-4a24-4c06-8e7e-3d8844987dd7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:12.029558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 818/1634] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:12.034260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:12.050366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22734908-25d3-4c71-aa1b-cda9827dcc5d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:12.067815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5366f0-6f0b-4ec0-a834-a626b4cf4ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:12.068415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5366f0-6f0b-4ec0-a834-a626b4cf4ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:12.068914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5366f0-6f0b-4ec0-a834-a626b4cf4ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:12.069379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5366f0-6f0b-4ec0-a834-a626b4cf4ea8 admin admin] PATH_INFO: `/v3/projects/4153e5736f014d42a74ed38f14028dd4/users/a5678b3f40f146c4aeee6ecaefb2f37c/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:12.078871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5366f0-6f0b-4ec0-a834-a626b4cf4ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4153e5736f014d42a74ed38f14028dd4, user_id=a5678b3f40f146c4aeee6ecaefb2f37c, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:12.084149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5366f0-6f0b-4ec0-a834-a626b4cf4ea8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:12.104182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 817/1635] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:12 2022] PUT /identity/v3/projects/4153e5736f014d42a74ed38f14028dd4/users/a5678b3f40f146c4aeee6ecaefb2f37c/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:12.176939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e4ec-03bc-4782-870d-f10cda2b6566 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:12.177411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e4ec-03bc-4782-870d-f10cda2b6566 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:12.177863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e4ec-03bc-4782-870d-f10cda2b6566 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:12.179690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 819/1636] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 17 21:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:12.187849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:12.188127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:12.188395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:12.223788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:12.225180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:12.328291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:12.388272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 818/1637] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:12.404182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547e4ec-03bc-4782-870d-f10cda2b6566 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:12.489767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:12.496052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547e4ec-03bc-4782-870d-f10cda2b6566 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:12.500745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8df6-fbeb-4106-bfab-1ea2ec1ba36a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:12.501452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8df6-fbeb-4106-bfab-1ea2ec1ba36a service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:12.501824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8df6-fbeb-4106-bfab-1ea2ec1ba36a service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:12.502162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8df6-fbeb-4106-bfab-1ea2ec1ba36a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:12.520841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb8df6-fbeb-4106-bfab-1ea2ec1ba36a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:12.523596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb8df6-fbeb-4106-bfab-1ea2ec1ba36a service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:12.541410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 820/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:12.574543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e1fc7-b9e4-4af8-88fb-0d36b9d3bcbf None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:12.581969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:12.583062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:12.583579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:12.584054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] PATH_INFO: `/v3/users/f7412e3b2d124a5994fc9ddb726b9865` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:12.585636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7412e3b2d124a5994fc9ddb726b9865)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:12.588148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:12.669460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7412e3b2d124a5994fc9ddb726b9865'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:12.682190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7412e3b2d124a5994fc9ddb726b9865'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:12.683449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7412e3b2d124a5994fc9ddb726b9865'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:12.694936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7412e3b2d124a5994fc9ddb726b9865'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:12.714506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7412e3b2d124a5994fc9ddb726b9865'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:12.723367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 819/1639] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:12 2022] DELETE /identity/v3/users/f7412e3b2d124a5994fc9ddb726b9865 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:12.730887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e874-93c6-48ee-a5fb-181f3a7b77b7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:12.731169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e874-93c6-48ee-a5fb-181f3a7b77b7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:12.731444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723e874-93c6-48ee-a5fb-181f3a7b77b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:12.765705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6723e874-93c6-48ee-a5fb-181f3a7b77b7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:12.766904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6723e874-93c6-48ee-a5fb-181f3a7b77b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:12.834936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:12.867592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 821/1640] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:12.934024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:12.935542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:13.071657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0351a45-2335-4d2f-a05f-113c46645f6f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:13.081705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029d63d-e5c1-4027-8b46-63d4557bfc4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:13.082298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029d63d-e5c1-4027-8b46-63d4557bfc4a service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:13.082669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029d63d-e5c1-4027-8b46-63d4557bfc4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:13.085064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029d63d-e5c1-4027-8b46-63d4557bfc4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:13.125047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029d63d-e5c1-4027-8b46-63d4557bfc4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:13.132492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029d63d-e5c1-4027-8b46-63d4557bfc4a service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:13.159060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 820/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:13.422510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2c34-7ccb-4516-9e90-72e0aaecf74d None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:13.423049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2c34-7ccb-4516-9e90-72e0aaecf74d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:13.423049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a2c34-7ccb-4516-9e90-72e0aaecf74d None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:13.424874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 822/1642] 10.222.0.16 () {54 vars in 965 bytes} [Sat Sep 17 21:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:13.434583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0daf7b-7469-4c51-9495-6fc84fa9da74 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:13.434888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0daf7b-7469-4c51-9495-6fc84fa9da74 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:13.435181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0daf7b-7469-4c51-9495-6fc84fa9da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:13.451172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:13.475773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5f0daf7b-7469-4c51-9495-6fc84fa9da74 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:13.485096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5f0daf7b-7469-4c51-9495-6fc84fa9da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:13.590348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 821/1643] 10.222.0.16 () {58 vars in 1062 bytes} [Sat Sep 17 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:13.597234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81020959-32be-4dc1-aa01-88e0e412549e None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:13.597708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81020959-32be-4dc1-aa01-88e0e412549e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:13.598063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81020959-32be-4dc1-aa01-88e0e412549e None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:13.599996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 823/1644] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 17 21:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 17 21:04:13.611522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ec2f-bcf6-4a21-b268-2d6fcb66b44d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:13.611956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ec2f-bcf6-4a21-b268-2d6fcb66b44d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:13.612374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597ec2f-bcf6-4a21-b268-2d6fcb66b44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:13.646105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2597ec2f-bcf6-4a21-b268-2d6fcb66b44d None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:13.648920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2597ec2f-bcf6-4a21-b268-2d6fcb66b44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:13.677494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 822/1645] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:13.678861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81020959-32be-4dc1-aa01-88e0e412549e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:13.680652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-81020959-32be-4dc1-aa01-88e0e412549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:13.756885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81020959-32be-4dc1-aa01-88e0e412549e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:13.763827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:13.764353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:13.764852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:13.765203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:13.779735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:13.781724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:13.794435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 824/1646] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:14.077828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:14.078327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:14.078327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:14.124233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] MFA Rules not processed for user `a5678b3f40f146c4aeee6ecaefb2f37c`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:14.126508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:14.242416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 823/1647] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:15.070018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:15.073649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:15.103653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8b560-d772-45f0-adfa-adc93a124eb7 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:15.116821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:15.117912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:15.119021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:15.119567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] PATH_INFO: `/v3/projects/3594924fe4fe453aa9a1c8f4c860779d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:15.122540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3594924fe4fe453aa9a1c8f4c860779d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:15.125204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:15.262600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3594924fe4fe453aa9a1c8f4c860779d', 'request_id': 'req-edb6213f-3418-4db7-91cb-58d710a39bff'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.271177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3594924fe4fe453aa9a1c8f4c860779d', 'request_id': 'req-edb6213f-3418-4db7-91cb-58d710a39bff'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.285921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3594924fe4fe453aa9a1c8f4c860779d', 'request_id': 'req-edb6213f-3418-4db7-91cb-58d710a39bff'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.321194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] The token cache is being invalidate because project 3594924fe4fe453aa9a1c8f4c860779d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:15.321490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.322166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-edb6213f-3418-4db7-91cb-58d710a39bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.324384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 825/1648] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:15 2022] DELETE /identity/v3/projects/3594924fe4fe453aa9a1c8f4c860779d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:15.328598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:15.344816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37897131-2f05-4958-92c2-10a2d479b1ff None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:15.434576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:15.435582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:15.436083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:15.436460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] PATH_INFO: `/v3/users/913661574f034136b64d3a05ef7c6c64` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:15.438537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=913661574f034136b64d3a05ef7c6c64)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:15.440823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:15.509620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:15.562748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913661574f034136b64d3a05ef7c6c64'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.577198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913661574f034136b64d3a05ef7c6c64'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.578955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913661574f034136b64d3a05ef7c6c64'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.591455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913661574f034136b64d3a05ef7c6c64'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.616759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-470128f9-acd7-4fa7-ab8f-4ef2ab8d0d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913661574f034136b64d3a05ef7c6c64'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:15.632882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 824/1649] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:15 2022] DELETE /identity/v3/users/913661574f034136b64d3a05ef7c6c64 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:16.157187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bfc0-f02f-41a0-b2c7-66e903000224 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:16.157715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bfc0-f02f-41a0-b2c7-66e903000224 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:16.157715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bfc0-f02f-41a0-b2c7-66e903000224 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:16.159072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 826/1650] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:16.182871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:16.183406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:16.183789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:16.234334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:16.236740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:16.457858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4bfc0-f02f-41a0-b2c7-66e903000224 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:16.460143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a7a4bfc0-f02f-41a0-b2c7-66e903000224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:16.511488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 825/1651] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:16.598997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4bfc0-f02f-41a0-b2c7-66e903000224 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:16.628717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:16.628717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:16.628717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:16.628717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] PATH_INFO: `/v3/users/dc3465a68b04477193d6c88130dab151` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:16.629169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3465a68b04477193d6c88130dab151)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:16.629169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: WARNING py.warnings [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:04:16.629169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: warnings.warn(msg) Sep 17 21:04:16.629169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: Sep 17 21:04:16.629169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:16.655290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:16.692881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:16.694610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:16.715878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38bd98-3c90-4455-9bb7-30971c08802d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:16.741478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b1179-6511-4a7a-8043-28b13119bd13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:16.742509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b1179-6511-4a7a-8043-28b13119bd13 service placement] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:16.743075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b1179-6511-4a7a-8043-28b13119bd13 service placement] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:16.743571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b1179-6511-4a7a-8043-28b13119bd13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:16.773382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b1179-6511-4a7a-8043-28b13119bd13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:16.775620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b1179-6511-4a7a-8043-28b13119bd13 service placement] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:16.810769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 826/1652] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:16.871166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3465a68b04477193d6c88130dab151'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:16.893850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3465a68b04477193d6c88130dab151'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:16.895229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3465a68b04477193d6c88130dab151'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:16.983488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3465a68b04477193d6c88130dab151'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.028737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3465a68b04477193d6c88130dab151'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.049378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 827/1653] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:16 2022] DELETE /identity/v3/users/dc3465a68b04477193d6c88130dab151 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:17.053615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:17.054626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:17.055001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:17.156611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:17.157047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:17.371665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 827/1654] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:17.388137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:17.390310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:17.434106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:17.435546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:17.447783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed9d9-89cf-4e53-a9c9-78173d55bdb6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:17.465666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:17.466645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:17.468764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:17.469700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] PATH_INFO: `/v3/projects/7ed6be4e5067451e983fc83572330989` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:17.471351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed6be4e5067451e983fc83572330989)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:17.484844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:17.527710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:17.649915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13721e39-7ee0-417c-b569-74ee6ddb6d6d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:17.692827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed6be4e5067451e983fc83572330989', 'request_id': 'req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.721730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed6be4e5067451e983fc83572330989', 'request_id': 'req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.742582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed6be4e5067451e983fc83572330989', 'request_id': 'req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.780954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] The token cache is being invalidate because project 7ed6be4e5067451e983fc83572330989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:17.783810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.785367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1e1e5b7f-cfdc-4425-b2ee-eed948c348ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:17.789516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 828/1655] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:17 2022] DELETE /identity/v3/projects/7ed6be4e5067451e983fc83572330989 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:17.796242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b559ac-e370-4785-ba3c-4b1ef8bb7abe None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:17.797090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b559ac-e370-4785-ba3c-4b1ef8bb7abe None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:17.797429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b559ac-e370-4785-ba3c-4b1ef8bb7abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:17.811136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:17.811839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:17.812204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:17.812542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:17.818181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:17.842620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-04b559ac-e370-4785-ba3c-4b1ef8bb7abe None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:17.844663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-04b559ac-e370-4785-ba3c-4b1ef8bb7abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:18.018896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 829/1656] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:18.020355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.020641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.020945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.049341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.051244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.071029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 828/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:18.081810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:18.091428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:18.125392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 830/1658] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:18.127316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:18.144553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9725fa4-2ff5-4873-8a29-2ca7986c3e4b service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:18.158394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:18.159034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.159698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.160597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.161557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:18.162036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.164792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.189143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:18.191591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 829/1659] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:18.194853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:18.205541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187425be-aa3a-4910-b094-2267f72218d3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:18.225938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:18.226586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.226931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.227209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.227867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.235084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.326499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 831/1660] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:18 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:18.331592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:18.355732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e4a88-4466-484c-81a4-1ed9010526d1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:18.541791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:18.541791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.541791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.541791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.542580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.543504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.599300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:18.609929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 830/1661] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:18 2022] POST /identity/v3/users => generated 284 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:18.615409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:18.634564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77b7b-ec7a-4872-a266-b79ed84c7eaf admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:18.647420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:18.664307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.664619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.666167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.671580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.678382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.707952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 832/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:18.712172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:18.739139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9478f-66ce-4ff2-b8c2-cde49614eaa5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:18.777651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d23c78-afa3-4752-984b-ace5961d8166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:18.778349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23c78-afa3-4752-984b-ace5961d8166 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.778563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23c78-afa3-4752-984b-ace5961d8166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.778839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23c78-afa3-4752-984b-ace5961d8166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.779498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d23c78-afa3-4752-984b-ace5961d8166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.781153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d23c78-afa3-4752-984b-ace5961d8166 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.827775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 831/1663] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:18.829681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:18.859821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a61727-5793-4f40-8a19-7934fcd2ba26 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:18.875901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:18.877231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.877740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.878226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] PATH_INFO: `/v3/projects/c99bec27a040441cadaf314869d75ed3/users/f419e1c03bc9440995842bbc9b706928/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:18.935424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c99bec27a040441cadaf314869d75ed3, user_id=f419e1c03bc9440995842bbc9b706928, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:18.938316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:18.987381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 833/1664] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:18 2022] PUT /identity/v3/projects/c99bec27a040441cadaf314869d75ed3/users/f419e1c03bc9440995842bbc9b706928/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:18.998522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:18.998809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:18.999077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:19.084765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] MFA Rules not processed for user `f419e1c03bc9440995842bbc9b706928`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:19.086441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:19.266813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 832/1665] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:19.271401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:19.409781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3ef90-f8ed-4b33-b8ef-d9f0f15635d1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:19.436796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:19.438183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:19.438657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:19.439533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:19.442333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:19.445046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:19.513918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 834/1666] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:19 2022] POST /identity/v3/projects => generated 367 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:19.515409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:19.534518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c6535-ffa3-4d98-be9a-bf64633683c0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:19.775611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:19.780068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:19.780373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:19.780660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:19.781424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:19.786443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:19.836217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 833/1667] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:19 2022] POST /identity/v3/users => generated 290 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:19.841788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:19.866051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac089a5-e6c4-4120-9407-3f6bc0c3ffde admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:19.890061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:19.891554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:19.894519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:19.894519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:19.895694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:19.899675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:19.954702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 835/1668] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:19.964693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:19.981827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56f855-8cf4-4158-a36a-50e8e858b91a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:19.998723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:19.999332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:19.999613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:19.999887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] PATH_INFO: `/v3/projects/635b27e7b0b045de8de2c83fd74eda52/users/e4814c89e2ed42f19b63357f9bd418e8/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.030172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=635b27e7b0b045de8de2c83fd74eda52, user_id=e4814c89e2ed42f19b63357f9bd418e8, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:20.032903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:20.071467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 834/1669] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:19 2022] PUT /identity/v3/projects/635b27e7b0b045de8de2c83fd74eda52/users/e4814c89e2ed42f19b63357f9bd418e8/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:20.079059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:20.113197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87573d9-98ff-4467-8e41-d76b844f3017 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:20.250266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3a19c-35b3-49ea-b2b9-6ec6965ec842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:20.250863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3a19c-35b3-49ea-b2b9-6ec6965ec842 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.251395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3a19c-35b3-49ea-b2b9-6ec6965ec842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.251809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3a19c-35b3-49ea-b2b9-6ec6965ec842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.252776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3a19c-35b3-49ea-b2b9-6ec6965ec842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:20.254515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3a19c-35b3-49ea-b2b9-6ec6965ec842 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:20.269492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 836/1670] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:20.273316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:20.287838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30df3-ec22-42db-aad3-45e41f50e8c8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:20.299258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:20.300099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.300473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.300903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] PATH_INFO: `/v3/projects/635b27e7b0b045de8de2c83fd74eda52/users/e4814c89e2ed42f19b63357f9bd418e8/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.303263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=635b27e7b0b045de8de2c83fd74eda52, user_id=e4814c89e2ed42f19b63357f9bd418e8, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:20.306466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:20.320399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 835/1671] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:20 2022] PUT /identity/v3/projects/635b27e7b0b045de8de2c83fd74eda52/users/e4814c89e2ed42f19b63357f9bd418e8/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:20.332915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.333526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.333885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.365641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:20.368827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:20.398507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] MFA Rules not processed for user `e4814c89e2ed42f19b63357f9bd418e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:20.401730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:20.418474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:20.453085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:20.534031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb380e70-fb6b-49f4-bcb2-efc46118d59f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:20.540382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 837/1672] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:20.549354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:20.549989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.550404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.550778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] PATH_INFO: `/v3/users/577fbc32bd544d228165ab9bedc5a14c` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.552008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=577fbc32bd544d228165ab9bedc5a14c)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:20.554340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:20.556938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:20.562533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:20.643892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577fbc32bd544d228165ab9bedc5a14c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:20.667365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577fbc32bd544d228165ab9bedc5a14c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:20.668511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577fbc32bd544d228165ab9bedc5a14c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:20.672263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3023cda-0ea2-400a-9704-3840cb81728b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:20.676772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:20.678457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.678835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.679678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.680561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577fbc32bd544d228165ab9bedc5a14c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:20.683182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:20.699860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:20.702063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:20.709054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577fbc32bd544d228165ab9bedc5a14c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:20.728190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 836/1673] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:20 2022] DELETE /identity/v3/users/577fbc32bd544d228165ab9bedc5a14c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:20.732118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 838/1674] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:04:20.737400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:20.738991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:20.794375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:20.798723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:20.818502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fe8be-6271-48df-8d39-fb6fcf74d2aa service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:20.886720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276e9c5-458c-4ace-92e5-9c0ab3535dca admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:20.894866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:20.895445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.895726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.896001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] PATH_INFO: `/v3/projects/9cdcf03c8c5e456b8f8ea08564ad6c94` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.898921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdcf03c8c5e456b8f8ea08564ad6c94)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:20.900134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: WARNING py.warnings [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 17 21:04:20.900134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: warnings.warn(msg) Sep 17 21:04:20.900134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: Sep 17 21:04:20.902781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:20.936396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:20.937076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:20.937326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:20.937602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:20.939438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:21.042705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdcf03c8c5e456b8f8ea08564ad6c94', 'request_id': 'req-0f008bec-6d1b-4531-9fb6-b37c5864f41e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.069668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:21.085165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdcf03c8c5e456b8f8ea08564ad6c94', 'request_id': 'req-0f008bec-6d1b-4531-9fb6-b37c5864f41e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.115981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdcf03c8c5e456b8f8ea08564ad6c94', 'request_id': 'req-0f008bec-6d1b-4531-9fb6-b37c5864f41e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.145091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:21.149761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] The token cache is being invalidate because project 9cdcf03c8c5e456b8f8ea08564ad6c94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:21.150052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.154090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a52c07-f0c0-45b1-8003-aebc58d5a98e service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:21.157565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0f008bec-6d1b-4531-9fb6-b37c5864f41e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:21.159808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 837/1675] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:20 2022] DELETE /identity/v3/projects/9cdcf03c8c5e456b8f8ea08564ad6c94 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:21.164084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:21.166067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:21.166347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:21.166683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:21.201521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 839/1676] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:21.246857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:21.250141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:21.420534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 838/1677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:21.421861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:21.422466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:21.422751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:21.464700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:21.464700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:21.496302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 840/1678] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:21.497378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:21.513843 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790a724-5dfe-499e-a65e-4e6d48fb09c3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:21.535224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:21.536078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:21.537070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:21.537534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:21.538313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:21.538753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:21.541279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:21.585371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 839/1679] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:21.590319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:21.611544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d802120-c5fd-4309-ba43-2a07fc86fdf6 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:21.624768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:21.626605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:21.629744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:21.631951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:21.633244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:21.637646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:21.716444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 841/1680] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:21 2022] POST /identity/v3/projects => generated 357 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:21.720643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:21.736715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba166e3-ebae-488b-b206-c3b8b5ed1041 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:21.950615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:21.951250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:21.951501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:21.951773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:21.952975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:21.954910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:22.005570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 840/1681] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:21 2022] POST /identity/v3/users => generated 279 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:22.009772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.026984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14230733-5807-47fd-8eda-59671c9e77b8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:22.040651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:22.042213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:22.042497 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:22.042773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:22.044593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:22.049835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:22.078127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.079775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 842/1682] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:22.094209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a48dfc-515f-4476-b799-30ffee132eb7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:22.107911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e613-5628-44a8-a300-1ee4b1df1774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:22.109864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e613-5628-44a8-a300-1ee4b1df1774 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:22.110156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e613-5628-44a8-a300-1ee4b1df1774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:22.110479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e613-5628-44a8-a300-1ee4b1df1774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:22.111149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731e613-5628-44a8-a300-1ee4b1df1774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:22.115262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731e613-5628-44a8-a300-1ee4b1df1774 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:22.140387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 841/1683] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:22.142120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.155830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a17a1-3031-477c-9533-a7fef15b6797 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:22.171407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:22.172095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:22.172377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:22.172844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] PATH_INFO: `/v3/projects/73263f8118e54a469e1c7500e3539e21/users/3ee759071bb64f43a8019fa13b34ec6b/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:22.218985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73263f8118e54a469e1c7500e3539e21, user_id=3ee759071bb64f43a8019fa13b34ec6b, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:22.224002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:22.259914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 843/1684] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:22 2022] PUT /identity/v3/projects/73263f8118e54a469e1c7500e3539e21/users/3ee759071bb64f43a8019fa13b34ec6b/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:22.271998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:22.272278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:22.272580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:22.333972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] MFA Rules not processed for user `3ee759071bb64f43a8019fa13b34ec6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:22.336415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:22.383735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.386703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:22.419035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:22.503592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 842/1685] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:22.504664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.570565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e8c68-4c37-4c39-a894-b670f9841c2b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:22.584627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:22.585648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:22.585930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:22.586206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] PATH_INFO: `/v3/projects/167bcd812954490e99cc625840fb5dd2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:22.591210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] RBAC: Authorizing `identity:delete_project(project_id=167bcd812954490e99cc625840fb5dd2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:22.597428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:22.677362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3406c-d69c-464a-8ed9-b890a37e3fdf None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:22.698809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:22.702546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:22.702778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:22.703062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:22.703700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:22.709285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:22.730195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '167bcd812954490e99cc625840fb5dd2', 'request_id': 'req-c4c103ea-9009-44ce-86e8-2d3f0884155b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:22.747766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '167bcd812954490e99cc625840fb5dd2', 'request_id': 'req-c4c103ea-9009-44ce-86e8-2d3f0884155b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:22.764496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '167bcd812954490e99cc625840fb5dd2', 'request_id': 'req-c4c103ea-9009-44ce-86e8-2d3f0884155b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:22.810069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 843/1686] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:22 2022] POST /identity/v3/projects => generated 357 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:22.817456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.818707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] The token cache is being invalidate because project 167bcd812954490e99cc625840fb5dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:22.819506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:22.821849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:22.828349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 844/1687] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:22 2022] DELETE /identity/v3/projects/167bcd812954490e99cc625840fb5dd2 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:22.834212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:22.841656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:22.911320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:23.003910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c103ea-9009-44ce-86e8-2d3f0884155b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.012294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b02f0-0ca1-46b4-a0ac-fca5763486bf admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.017115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.017720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.017836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.018033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.018300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] PATH_INFO: `/v3/users/a5678b3f40f146c4aeee6ecaefb2f37c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.018396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.018781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.019582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.020333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.022274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5678b3f40f146c4aeee6ecaefb2f37c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.022912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.023675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.101768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 844/1688] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:22 2022] POST /identity/v3/users => generated 285 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:23.105258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.111602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:23.148515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ba6ec-b854-4367-b08b-5e80ff3e4fb9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.162467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.165207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.165505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.167389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.169423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.181081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5678b3f40f146c4aeee6ecaefb2f37c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.185286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.205157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5678b3f40f146c4aeee6ecaefb2f37c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.212730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5678b3f40f146c4aeee6ecaefb2f37c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.223696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 845/1689] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:23.224939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.226935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:23.245114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cb9dd-2046-4465-85ac-89688c8d5e98 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.253070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5678b3f40f146c4aeee6ecaefb2f37c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.263021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.264134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.264489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.264697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] PATH_INFO: `/v3/projects/c825198689d74640b9d7c4f1678af2c3/users/024ee409ec3b43a6b622ed4573cc3a72/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.299024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5678b3f40f146c4aeee6ecaefb2f37c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:23.303448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c825198689d74640b9d7c4f1678af2c3, user_id=024ee409ec3b43a6b622ed4573cc3a72, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.310777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.318458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 845/1690] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:22 2022] DELETE /identity/v3/users/a5678b3f40f146c4aeee6ecaefb2f37c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:23.341845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 846/1691] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:23 2022] PUT /identity/v3/projects/c825198689d74640b9d7c4f1678af2c3/users/024ee409ec3b43a6b622ed4573cc3a72/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:23.346122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.554916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a9af9-6b3f-4154-b9ac-902a80e7154e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.716633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:23.742141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ed12d-811e-40d6-b59a-d1eec8a07325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.742818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ed12d-811e-40d6-b59a-d1eec8a07325 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.743073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ed12d-811e-40d6-b59a-d1eec8a07325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.745780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ed12d-811e-40d6-b59a-d1eec8a07325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.746450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ed12d-811e-40d6-b59a-d1eec8a07325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.748106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ed12d-811e-40d6-b59a-d1eec8a07325 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.777723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 846/1692] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:23.781970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:23.807491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b7581-25a5-40af-8779-33c838090d2b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:23.837425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:23.838686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.842926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.842926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] PATH_INFO: `/v3/projects/c825198689d74640b9d7c4f1678af2c3/users/024ee409ec3b43a6b622ed4573cc3a72/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:23.853167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c825198689d74640b9d7c4f1678af2c3, user_id=024ee409ec3b43a6b622ed4573cc3a72, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:23.857872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:23.905488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 847/1693] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:23 2022] PUT /identity/v3/projects/c825198689d74640b9d7c4f1678af2c3/users/024ee409ec3b43a6b622ed4573cc3a72/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:23.915992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:23.917398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:23.917966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.013247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] MFA Rules not processed for user `024ee409ec3b43a6b622ed4573cc3a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.019183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.150610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.154009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.162873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:24.227938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 847/1694] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:24.240972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.242705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.298414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d8a6-9f5c-4da8-bfb1-7f9372cdaee5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.313473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.335222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.335222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.335222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] PATH_INFO: `/v3/projects/4153e5736f014d42a74ed38f14028dd4` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.340283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4153e5736f014d42a74ed38f14028dd4)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.343322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.404364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3b226-0181-46c4-930c-0674ee4ecadb None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.419692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.420325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.420722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.420997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.437848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:24.439348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:24.463509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 848/1695] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:24.496389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4153e5736f014d42a74ed38f14028dd4', 'request_id': 'req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.507013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4153e5736f014d42a74ed38f14028dd4', 'request_id': 'req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.536330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4153e5736f014d42a74ed38f14028dd4', 'request_id': 'req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.576392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] The token cache is being invalidate because project 4153e5736f014d42a74ed38f14028dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:24.576927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.578908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c41b1101-a94f-4e60-9a50-e4e3a6b2924a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:24.584228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:24.587860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 848/1696] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:24 2022] DELETE /identity/v3/projects/4153e5736f014d42a74ed38f14028dd4 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:24.591895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecf393-ec90-4c19-b94f-80ff3a3d67d7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.592190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecf393-ec90-4c19-b94f-80ff3a3d67d7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.592788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecf393-ec90-4c19-b94f-80ff3a3d67d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.597048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754f599-db24-4627-ab46-4ab6629ebc18 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:24.656527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-dfecf393-ec90-4c19-b94f-80ff3a3d67d7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.660329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-dfecf393-ec90-4c19-b94f-80ff3a3d67d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.719865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:24.720496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.720841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.721113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.730715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.862196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 849/1697] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:24.872342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:24.872638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:24.873013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:24.904973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:24.955896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:24.960269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:24.999220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.000949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.004420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 850/1698] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:25.005770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.022997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d8acd-db91-447f-9f92-402f2f4d99a0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.041582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 849/1699] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:04:25.044542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.045354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.045641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.045918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.046585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:25.046887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.048892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.081921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.083990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 851/1700] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:04:25.108474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfe3f8-f1aa-4208-8763-9f3f2104d0cc service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.132950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.133553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.133830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.134102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.134745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.145509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.181476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.184251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:25.218625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d74c8-330d-4c1b-935e-24b2ce694d4a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.233351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 850/1701] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:25 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:25.234998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.235144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.235816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.238296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.238615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] PATH_INFO: `/v3/users/4784fbffe44e463f9cb71c0195ced4d9` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.241900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4784fbffe44e463f9cb71c0195ced4d9)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.249157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.266333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:25.438229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4784fbffe44e463f9cb71c0195ced4d9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:25.459169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4784fbffe44e463f9cb71c0195ced4d9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:25.460125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4784fbffe44e463f9cb71c0195ced4d9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:25.496837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4784fbffe44e463f9cb71c0195ced4d9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:25.534783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b04a-afbe-40af-9041-21064ac3a26e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.555381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-c7b53155-c23a-4d6a-8c67-f2bb29d2ae0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4784fbffe44e463f9cb71c0195ced4d9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:25.558368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.563288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.565089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.565269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.575416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.583566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.591062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 852/1702] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:25 2022] DELETE /identity/v3/users/4784fbffe44e463f9cb71c0195ced4d9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:25.593014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa56a-faf0-4fa0-95e6-ff46506f43a0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.593306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa56a-faf0-4fa0-95e6-ff46506f43a0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.593584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fa56a-faf0-4fa0-95e6-ff46506f43a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.638136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 851/1703] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:25 2022] POST /identity/v3/users => generated 291 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:25.639788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.643316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:25.686215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84acb6-8f39-422a-a5e1-0efad62121c0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.687475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-693fa56a-faf0-4fa0-95e6-ff46506f43a0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:25.688785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-693fa56a-faf0-4fa0-95e6-ff46506f43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:25.898562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 853/1704] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:25.898972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.902712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.903017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.903320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.906706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.907210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae3d3-9fdd-4050-849c-b4429dbabeb4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.907826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae3d3-9fdd-4050-849c-b4429dbabeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.909029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae3d3-9fdd-4050-849c-b4429dbabeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.910179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.939103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 852/1705] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:25.940884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:25.961687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59114e21-d9a5-4b99-82cd-c898abbf5c57 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:25.977233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:25.977864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:25.978144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:25.978553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:25.982961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:25.984930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:25.994328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e89ae3d3-9fdd-4050-849c-b4429dbabeb4 None None] MFA Rules not processed for user `a3dac44116784a9e963a69e4793bac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:25.996413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e89ae3d3-9fdd-4050-849c-b4429dbabeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:26.013292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 853/1706] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:26.015503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:26.037450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342536c0-c711-453e-8343-fb29414939b7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:26.054820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:26.056275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:26.056763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:26.057703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] PATH_INFO: `/v3/projects/2a8db2f14a6f4721aff71d14087c841b/users/2e15b43261ca4481a3e6f27ab1ef1e38/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:26.092618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8db2f14a6f4721aff71d14087c841b, user_id=2e15b43261ca4481a3e6f27ab1ef1e38, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:26.097166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:26.127346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:26.150704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 854/1707] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:26 2022] PUT /identity/v3/projects/2a8db2f14a6f4721aff71d14087c841b/users/2e15b43261ca4481a3e6f27ab1ef1e38/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:26.153285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:26.157517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:26.265400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 854/1708] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:26.267694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6787f9-57ff-461e-ab46-b2f7a5f25145 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:26.267694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6787f9-57ff-461e-ab46-b2f7a5f25145 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:26.267852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6787f9-57ff-461e-ab46-b2f7a5f25145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:26.338536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7cd1c-5386-4461-8e5f-fcf517b4133c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:26.345789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:26.346500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:26.346772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:26.347102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:26.383281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ad6787f9-57ff-461e-ab46-b2f7a5f25145 None None] MFA Rules not processed for user `2e15b43261ca4481a3e6f27ab1ef1e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:26.384647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ad6787f9-57ff-461e-ab46-b2f7a5f25145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:26.394623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:26.550527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:26.552124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:26.582992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 855/1709] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:26.590723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:26.595099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:26.595425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:26.598242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 855/1710] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:26.603729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:26.639608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b6366b-2534-423c-ba07-249af481346a service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:26.670049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:26.671097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] MFA Rules not processed for user `a3dac44116784a9e963a69e4793bac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:26.672514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:26.692498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:26.693146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:26.693571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:26.694388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:26.699916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:26.772133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 856/1711] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:26 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:26.776682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:26.865864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 856/1712] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:26.876569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:26.879001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:26.929695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bb591-d8bd-4fa2-b75b-fb32e60d0dad None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:26.990468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bbb80-8d7e-489a-afd7-25f542a31e4c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:27.012732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:27.013365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.013649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.013913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.014557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:27.022889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:27.081722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 857/1713] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:26 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:27.084206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:27.097394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.112241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:27.113893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.114628 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.115434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] PATH_INFO: `/v3/users/f419e1c03bc9440995842bbc9b706928` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.121488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f419e1c03bc9440995842bbc9b706928)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:27.125555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:27.148290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:27.270011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f419e1c03bc9440995842bbc9b706928'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:27.277032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:27.292494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f419e1c03bc9440995842bbc9b706928'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:27.293614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f419e1c03bc9440995842bbc9b706928'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:27.313880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330af480-bd49-450b-8cfa-5aafedde17dc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:27.314544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f419e1c03bc9440995842bbc9b706928'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:27.330672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:27.331265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.331537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.331898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.341425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.353430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-eb239d6e-9935-4373-912f-be090f38ced5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f419e1c03bc9440995842bbc9b706928'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:27.386544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 857/1714] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:26 2022] DELETE /identity/v3/users/f419e1c03bc9440995842bbc9b706928 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:27.398533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d76e1-a76d-4fd0-8d78-7539ffd63df4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.400455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d76e1-a76d-4fd0-8d78-7539ffd63df4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.400455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d76e1-a76d-4fd0-8d78-7539ffd63df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.482047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1f6d76e1-a76d-4fd0-8d78-7539ffd63df4 None None] MFA Rules not processed for user `1fa4420540af41e4967b0f731f818af5`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:27.483602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1f6d76e1-a76d-4fd0-8d78-7539ffd63df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.529775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:27.531197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:27.556769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 858/1715] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:27.558283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:27.568177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.608465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bbc60-2d57-4291-8bb6-4be220bdea97 service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:27.726545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 858/1716] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:27.738721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.738721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.738721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.784515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:27.786235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.802634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fed782-ca3d-4bab-bdc2-31047c3274e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:27.803276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed782-ca3d-4bab-bdc2-31047c3274e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.803555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed782-ca3d-4bab-bdc2-31047c3274e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.803830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fed782-ca3d-4bab-bdc2-31047c3274e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.804464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fed782-ca3d-4bab-bdc2-31047c3274e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:27.814408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fed782-ca3d-4bab-bdc2-31047c3274e5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:27.849245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 859/1717] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:27.856012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.856322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.856584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.902129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] MFA Rules not processed for user `1fa4420540af41e4967b0f731f818af5`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:27.903365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:27.931117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 860/1718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:27.932266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:27.958357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bec32-a81d-467b-ae89-e1501f616179 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:27.971727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 859/1719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:27.976931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:27.977376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:27.978062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:27.978337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:27.978613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] PATH_INFO: `/v3/projects/7272b73ad1ff43e4a0ee0881b4d40411/users/e9e36ab4a76843f48723e3cd69a9b8af/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:27.979510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:28.022270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7272b73ad1ff43e4a0ee0881b4d40411, user_id=e9e36ab4a76843f48723e3cd69a9b8af, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:28.025192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:28.051876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 861/1720] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:27 2022] PUT /identity/v3/projects/7272b73ad1ff43e4a0ee0881b4d40411/users/e9e36ab4a76843f48723e3cd69a9b8af/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:28.055921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:28.060639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:28.086002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de44426-da16-4d72-aebb-cdd7a833ee6b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:28.126891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:28.237868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16464f-3f53-490d-a155-9ed156318415 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:28.242693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:28.243316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:28.243610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:28.243868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:28.244760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:28.246445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:28.287280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 862/1721] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:28.292507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:28.299494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:28.334739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8378ecbc-5e00-412d-b7d5-470f32fba23b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:28.336438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378ecbc-5e00-412d-b7d5-470f32fba23b service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:28.336816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378ecbc-5e00-412d-b7d5-470f32fba23b service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:28.337154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8378ecbc-5e00-412d-b7d5-470f32fba23b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:28.472595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af7d65-2722-4d59-8f72-a047b3719f3e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:28.489121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:28.489826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:28.490011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:28.490290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:28.517860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:28.523801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:28.531579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378ecbc-5e00-412d-b7d5-470f32fba23b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:28.533266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8378ecbc-5e00-412d-b7d5-470f32fba23b service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:28.553348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 863/1722] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:28.554236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:28.564993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 860/1723] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:28.585569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5733f21-fbe9-4ca7-ae23-372afcff9574 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:28.595071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:28.595435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:28.596930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:28.597285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] PATH_INFO: `/v3/projects/7272b73ad1ff43e4a0ee0881b4d40411/users/e9e36ab4a76843f48723e3cd69a9b8af/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:28.604417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7272b73ad1ff43e4a0ee0881b4d40411, user_id=e9e36ab4a76843f48723e3cd69a9b8af, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:28.614361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:28.640992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 864/1724] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:28 2022] PUT /identity/v3/projects/7272b73ad1ff43e4a0ee0881b4d40411/users/e9e36ab4a76843f48723e3cd69a9b8af/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:28.647124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:28.647556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:28.647836 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:28.731473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] MFA Rules not processed for user `e9e36ab4a76843f48723e3cd69a9b8af`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:28.735859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:28.891215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:28.893171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:28.924292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 861/1725] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:28.940898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:28.943680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:28.999371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:29.099674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fddc8-375c-4812-953d-fd7cebef83b4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:29.104478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a72b7-dd5d-4847-be42-2cbf083921ea admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:29.113552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:29.114564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.115011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.115427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] PATH_INFO: `/v3/projects/1265ca6c66b24a0e9a98331fb134c262` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.124618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1265ca6c66b24a0e9a98331fb134c262)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:29.131743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:29.132354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3699077-2726-4f2a-ba41-98eee15b61cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:29.132859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699077-2726-4f2a-ba41-98eee15b61cb service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.133145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699077-2726-4f2a-ba41-98eee15b61cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.133420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3699077-2726-4f2a-ba41-98eee15b61cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.157669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699077-2726-4f2a-ba41-98eee15b61cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:29.159033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3699077-2726-4f2a-ba41-98eee15b61cb service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:29.195936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 862/1726] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:29.267424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:29.268830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1265ca6c66b24a0e9a98331fb134c262', 'request_id': 'req-bb896636-c78c-45b0-b83d-5086b4969a60'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:29.274959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1265ca6c66b24a0e9a98331fb134c262', 'request_id': 'req-bb896636-c78c-45b0-b83d-5086b4969a60'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:29.303776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1265ca6c66b24a0e9a98331fb134c262', 'request_id': 'req-bb896636-c78c-45b0-b83d-5086b4969a60'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:29.326146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf1f30-e9ca-4bf4-99ab-786a6d9f7881 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.326146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf1f30-e9ca-4bf4-99ab-786a6d9f7881 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.326146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf1f30-e9ca-4bf4-99ab-786a6d9f7881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.335893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] The token cache is being invalidate because project 1265ca6c66b24a0e9a98331fb134c262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:29.336735 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:29.337566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bb896636-c78c-45b0-b83d-5086b4969a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:29.339922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 865/1727] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:28 2022] DELETE /identity/v3/projects/1265ca6c66b24a0e9a98331fb134c262 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:29.345684 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.346274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.346770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.373527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:29.375341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:29.385330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-56bf1f30-e9ca-4bf4-99ab-786a6d9f7881 None None] MFA Rules not processed for user `4b5e98fb43a143ea8cfa30990b4fffc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:29.387933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-56bf1f30-e9ca-4bf4-99ab-786a6d9f7881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:29.461172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 866/1728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:29.462110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:29.464400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:29.634166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 863/1729] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:29.639995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc3b0e-d2b6-4d4a-ad7a-4add1ec03493 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.641705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc3b0e-d2b6-4d4a-ad7a-4add1ec03493 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.642199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc3b0e-d2b6-4d4a-ad7a-4add1ec03493 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.650058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 864/1730] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 17 21:04:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:29.655427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bf6bf-7000-43b4-b257-f378380b6654 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.655890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bf6bf-7000-43b4-b257-f378380b6654 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.657910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bf6bf-7000-43b4-b257-f378380b6654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.737980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2b5bf6bf-7000-43b4-b257-f378380b6654 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:29.740346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2b5bf6bf-7000-43b4-b257-f378380b6654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:29.746983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e60a6-7f4f-4517-b5ad-639352f27be3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:29.758545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:29.761139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.761472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.761754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.766002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:29.788074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 865/1731] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:29.795320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.800098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.800621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.907866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:29.909524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523bdd1-efb8-4db7-bb28-9a5b385cb728 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:29.930559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 867/1732] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:29.934100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92109d4-4219-476f-9c6a-e4212ae02ae7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:29.934389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92109d4-4219-476f-9c6a-e4212ae02ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:29.934670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92109d4-4219-476f-9c6a-e4212ae02ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:29.965512 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] MFA Rules not processed for user `1e3b1e0f4888400b89c47b3e44443228`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:29.966740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.062011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e92109d4-4219-476f-9c6a-e4212ae02ae7 None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:30.063488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e92109d4-4219-476f-9c6a-e4212ae02ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.087967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:30.187670 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 866/1733] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:30.188663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.204244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 868/1734] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:30.206122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880b40-a0bb-48b9-a4fc-0fdc74816fc3 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.206553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880b40-a0bb-48b9-a4fc-0fdc74816fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.206927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880b40-a0bb-48b9-a4fc-0fdc74816fc3 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.215358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 869/1735] 10.222.0.16 () {54 vars in 912 bytes} [Sat Sep 17 21:04:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:30.216622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880b40-a0bb-48b9-a4fc-0fdc74816fc3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.222595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-46880b40-a0bb-48b9-a4fc-0fdc74816fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.231833 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc5fa8-48f6-4098-91e5-3c3b0f3a2897 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.245470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.247199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.247495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.247800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.257483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:30.258397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.260258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72cd13-eb1d-4f74-b78c-ecd40c44bd5f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.260781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880b40-a0bb-48b9-a4fc-0fdc74816fc3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.274917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 867/1736] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:30.277398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.278003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.278287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.281127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.281549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.281720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.297353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.310953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:30.312252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.329533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.332235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.351367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 868/1737] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:30.352319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.354474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.362639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 870/1738] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:30.371210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.390912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efd0b7-50cd-47dc-bc3f-bf4954035bfa None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.407016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.407951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.408305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.410692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] PATH_INFO: `/v3/projects/c99bec27a040441cadaf314869d75ed3` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.413946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8c492-0ca4-4a04-9398-fd958884b41e service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.414354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c99bec27a040441cadaf314869d75ed3)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.421640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.435082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.436137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.436137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.436284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.436859 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.438806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.472969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:30.529550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 871/1739] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:30 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:30.529550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.582547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c99bec27a040441cadaf314869d75ed3', 'request_id': 'req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:30.599466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c99bec27a040441cadaf314869d75ed3', 'request_id': 'req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:30.631893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c99bec27a040441cadaf314869d75ed3', 'request_id': 'req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:30.672430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] The token cache is being invalidate because project c99bec27a040441cadaf314869d75ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:30.673074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:30.674637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:30.682815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 869/1740] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:30 2022] DELETE /identity/v3/projects/c99bec27a040441cadaf314869d75ed3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:30.683006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d641e-62a5-48d0-b0c8-73c1634f3416 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.683894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.688010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.784386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.785760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.785760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.785760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.787379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.810014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.811500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.831797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 872/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:30.832333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.838284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:30.848538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba586bff-80c0-4e64-94a7-fa2d9f8b7ff1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.857869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.858503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.858784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.859059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.859945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.863448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.875364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a990679-50df-406d-a08d-8ccd6ab3669a service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.881514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.882168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.882514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.882959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] PATH_INFO: `/v3/users/e4814c89e2ed42f19b63357f9bd418e8` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.888635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4814c89e2ed42f19b63357f9bd418e8)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.890700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:30.920148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 870/1742] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:30 2022] POST /identity/v3/users => generated 288 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:30.924266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:30.970054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc781a7-83a2-4de8-a410-0c0194767983 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:30.984239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:30.986794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:30.987157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:30.988821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:30.990597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:30.993841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:31.019329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:31.024761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 871/1743] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:31.026741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4814c89e2ed42f19b63357f9bd418e8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:31.027638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:31.044240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4814c89e2ed42f19b63357f9bd418e8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:31.044962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4814c89e2ed42f19b63357f9bd418e8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:31.060849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c7fe-4685-4bdf-8111-1bee1ebaf773 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:31.067037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4814c89e2ed42f19b63357f9bd418e8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:31.067855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:31.070137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:31.070225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:31.071868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:31.075648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:31.077763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:31.078592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4814c89e2ed42f19b63357f9bd418e8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:31.096498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 873/1744] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:30 2022] DELETE /identity/v3/users/e4814c89e2ed42f19b63357f9bd418e8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:31.109327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 872/1745] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:31.114233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:31.115778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:31.230106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e750a54-7b77-4490-88bc-8e238419008d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:31.269169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4a0c4-5781-4a3d-ab19-c11b268f0ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:31.280836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a0c4-5781-4a3d-ab19-c11b268f0ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:31.284594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:31.286853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:31.298288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a0c4-5781-4a3d-ab19-c11b268f0ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:31.301646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4a0c4-5781-4a3d-ab19-c11b268f0ba8 admin admin] PATH_INFO: `/v3/projects/acb8cf4cfdc242ef81676957e2b81a56/users/7b5cc9065d6e45398ceb0b8a77b4f8d3/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:31.414141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a0c4-5781-4a3d-ab19-c11b268f0ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb8cf4cfdc242ef81676957e2b81a56, user_id=7b5cc9065d6e45398ceb0b8a77b4f8d3, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:31.417480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4a0c4-5781-4a3d-ab19-c11b268f0ba8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:31.458592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 874/1746] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:31 2022] PUT /identity/v3/projects/acb8cf4cfdc242ef81676957e2b81a56/users/7b5cc9065d6e45398ceb0b8a77b4f8d3/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:31.466873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:31.468051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:31.468918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:31.614528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:31.623519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cbece-9ebf-4b46-b788-4c75cb50ec74 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:31.626766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] MFA Rules not processed for user `7b5cc9065d6e45398ceb0b8a77b4f8d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:31.636215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:31.876439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:31.878393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:31.878685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:31.878964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:31.899369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:31.916289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 875/1747] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:31.919643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:31.923419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.042689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:32.044322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:32.060953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:32.061931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 873/1748] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:32.065257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:32.078231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.117465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad79b35-ca50-4330-a7b7-71a499d5f9e1 service nova] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:32.137239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:32.139075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.139364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.140777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.146397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.172422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21948c2-b874-4662-b53e-e258781a1eba None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:32.196271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:32.196845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.197997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.198408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.215640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.273310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:32.274883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872026a-352c-4688-8c3b-b7620560fe06 service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:32.324696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 876/1749] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:32.324696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999db22-5799-443d-8b20-6c3d4809ae81 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.324696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999db22-5799-443d-8b20-6c3d4809ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.324696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999db22-5799-443d-8b20-6c3d4809ae81 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.340178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 877/1750] 10.222.0.16 () {54 vars in 961 bytes} [Sat Sep 17 21:04:32 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:04:32.342677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999db22-5799-443d-8b20-6c3d4809ae81 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:32.348510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7999db22-5799-443d-8b20-6c3d4809ae81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.359365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:32.365430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8284a6a-62f4-49d5-ae1f-a02d4a615e9d service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:32.390975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 874/1751] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:32.394267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.395407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.397328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.476268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:32.477548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.519518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 875/1752] 10.222.0.16 () {58 vars in 1058 bytes} [Sat Sep 17 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:32.520506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:32.555393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e02011-f249-4623-a81c-78991093c1f5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:32.581282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f3fdf-ea9b-4379-acf3-284ed7d04404 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:32.581912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3fdf-ea9b-4379-acf3-284ed7d04404 service placement] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.582712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3fdf-ea9b-4379-acf3-284ed7d04404 service placement] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.582712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3fdf-ea9b-4379-acf3-284ed7d04404 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.601917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f3fdf-ea9b-4379-acf3-284ed7d04404 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:32.605327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f3fdf-ea9b-4379-acf3-284ed7d04404 service placement] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:32.641990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 876/1753] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:32.674320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999db22-5799-443d-8b20-6c3d4809ae81 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:32.697958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:32.698874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.699155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.699436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] PATH_INFO: `/v3/projects/635b27e7b0b045de8de2c83fd74eda52` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.708713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] RBAC: Authorizing `identity:delete_project(project_id=635b27e7b0b045de8de2c83fd74eda52)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:32.711146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:32.808866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd791cdf-2327-4ac6-9ccf-d4378dd009c9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:32.809595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd791cdf-2327-4ac6-9ccf-d4378dd009c9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:32.809595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd791cdf-2327-4ac6-9ccf-d4378dd009c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:32.843192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '635b27e7b0b045de8de2c83fd74eda52', 'request_id': 'req-e15e5ffa-8dda-4ac9-81fc-e96012f943af'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:32.864496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bd791cdf-2327-4ac6-9ccf-d4378dd009c9 None None] MFA Rules not processed for user `a3dac44116784a9e963a69e4793bac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:32.865275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '635b27e7b0b045de8de2c83fd74eda52', 'request_id': 'req-e15e5ffa-8dda-4ac9-81fc-e96012f943af'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:32.865888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bd791cdf-2327-4ac6-9ccf-d4378dd009c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:32.886438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '635b27e7b0b045de8de2c83fd74eda52', 'request_id': 'req-e15e5ffa-8dda-4ac9-81fc-e96012f943af'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:32.934296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:32.938550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] The token cache is being invalidate because project 635b27e7b0b045de8de2c83fd74eda52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:32.938920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:32.952562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:32.961391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 878/1754] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:32 2022] DELETE /identity/v3/projects/635b27e7b0b045de8de2c83fd74eda52 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:32.975114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:32.989389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:33.141327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 877/1755] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:33.142801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf704efe-189d-4bdc-9499-34878d0dde1e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:33.143240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf704efe-189d-4bdc-9499-34878d0dde1e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:33.157972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf704efe-189d-4bdc-9499-34878d0dde1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:33.253639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bf704efe-189d-4bdc-9499-34878d0dde1e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:33.262735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bf704efe-189d-4bdc-9499-34878d0dde1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:33.344000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:33.418793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5ffa-8dda-4ac9-81fc-e96012f943af admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:33.442115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:33.442882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:33.443346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:33.444406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:33.451347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:33.566306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 878/1756] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:33.589864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:33.591110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:33.591510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:33.688226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:33.692561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:33.693579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:33.695308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:33.725348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 879/1757] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:33.729930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:33.736040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 879/1758] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:33.754850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b453e-88e9-4136-b084-13d86b6b3489 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:33.762756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:33.765226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:33.765226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:33.765226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:33.765226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:33.765226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:33.767746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:33.805987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 880/1759] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:33.811592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:33.827266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e7f9-57df-47a2-84b1-57fe4ecfeb64 service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:33.838880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:33.839315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:33.839627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:33.841215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:33.843008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:33.849104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:33.928405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:33.929179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 880/1760] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:33 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:33.952898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536774c3-eb30-4755-a38c-3a14bb84dc31 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:34.221619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:34.223154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.223444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.226497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.230252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:34.236017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:34.325160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 881/1761] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:33 2022] POST /identity/v3/users => generated 293 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:34.329661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:34.347173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a398c-810f-4838-89ab-c1f8ac8b6566 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:34.391966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:34.399831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.401105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.402819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.404842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:34.414424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:34.481261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 881/1762] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:34.481804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:34.513990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba77e5f-9879-47b7-84ee-82f5e21099bf admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:34.538731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:34.542655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.546166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.554807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.555979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:34.561188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:34.608349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 882/1763] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:34.612123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:34.630472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88565e1b-a6a0-4bfe-aac3-f5a4e6c95162 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:34.641936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e2ee9-d5a4-4466-b833-165e9958bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:34.642682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e2ee9-d5a4-4466-b833-165e9958bbba admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.642949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e2ee9-d5a4-4466-b833-165e9958bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.643212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e2ee9-d5a4-4466-b833-165e9958bbba admin admin] PATH_INFO: `/v3/projects/333dddd51a5e4af89f8cfee915093ba3/users/c615664e3b2f49cdbd1ee341cf82b647/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.687021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:34.689492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:34.701118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e2ee9-d5a4-4466-b833-165e9958bbba admin admin] RBAC: Authorizing `identity:create_grant(project_id=333dddd51a5e4af89f8cfee915093ba3, user_id=c615664e3b2f49cdbd1ee341cf82b647, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:34.708259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e2ee9-d5a4-4466-b833-165e9958bbba admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:34.752970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 882/1764] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:34 2022] PUT /identity/v3/projects/333dddd51a5e4af89f8cfee915093ba3/users/c615664e3b2f49cdbd1ee341cf82b647/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:34.754891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830c702-ab4e-4a74-8ace-f65434ccd78c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:34.763478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.764111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.766047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.810672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] MFA Rules not processed for user `c615664e3b2f49cdbd1ee341cf82b647`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:34.812281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:34.847109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:34.953527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:34.955540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:34.957716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:34.958279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] PATH_INFO: `/v3/users/7b5cc9065d6e45398ceb0b8a77b4f8d3` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:34.962026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5cc9065d6e45398ceb0b8a77b4f8d3)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:34.963360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 883/1765] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:34.967339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:34.969034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:35.037014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf266b-ffdc-4345-904d-e41c2d3b941a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:35.067467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:35.069654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:35.073606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:35.074237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:35.082648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:35.090306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:35.136545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5cc9065d6e45398ceb0b8a77b4f8d3'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:35.160788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5cc9065d6e45398ceb0b8a77b4f8d3'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:35.169696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5cc9065d6e45398ceb0b8a77b4f8d3'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:35.171910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 884/1766] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:34 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:35.173797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:35.184998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:35.193662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:35.205056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5cc9065d6e45398ceb0b8a77b4f8d3'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:35.237914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b39c33cf-b90a-41ce-88a9-1a3202ef0b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5cc9065d6e45398ceb0b8a77b4f8d3'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:35.255762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 883/1767] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:34 2022] DELETE /identity/v3/users/7b5cc9065d6e45398ceb0b8a77b4f8d3 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:35.257852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:35.258145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:35.258424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:35.308841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:35.310110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:35.330987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630321d-7032-46bb-83e0-9b298b91ece8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:35.461258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 884/1768] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:35.527727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:35.528954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:35.530215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:35.531146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:35.533286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:35.539422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:35.543835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:35.545767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:35.647192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 885/1769] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:35 2022] POST /identity/v3/users => generated 299 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:35.653998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:35.654149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:35.679395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf5c3a-5138-45eb-b0dd-e67f9206b44c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:35.680635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2478d0cc-cded-4fe4-b985-188bdaa8cf81 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:35.690909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:35.691622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:35.691849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:35.692087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:35.705012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:35.705838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:35.705947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:35.706173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:35.706832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:35.710890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:35.711427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:35.713299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:35.740895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 886/1770] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:35.742500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 885/1771] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:35.747785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:35.774323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8184a-07c0-401b-87d7-67b7accfd96f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:35.797877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:35.798669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:35.799260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:35.800760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] PATH_INFO: `/v3/projects/9df41e76392649f1aebd9c28476d1760/users/20f817e074ac4f00a1ad36994314d118/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:35.846663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df41e76392649f1aebd9c28476d1760, user_id=20f817e074ac4f00a1ad36994314d118, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:35.852312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:35.883048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 887/1772] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:35 2022] PUT /identity/v3/projects/9df41e76392649f1aebd9c28476d1760/users/20f817e074ac4f00a1ad36994314d118/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:35.884958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:35.898030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b58991-0a31-45e9-8208-e18de575cacf service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:36.031294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67daf06-f394-45c7-a7a4-a9cfaf5bb243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:36.032340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67daf06-f394-45c7-a7a4-a9cfaf5bb243 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:36.032877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67daf06-f394-45c7-a7a4-a9cfaf5bb243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:36.033294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67daf06-f394-45c7-a7a4-a9cfaf5bb243 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:36.034119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67daf06-f394-45c7-a7a4-a9cfaf5bb243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:36.035938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67daf06-f394-45c7-a7a4-a9cfaf5bb243 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:36.083790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 886/1773] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:36.087137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:36.106300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd1c7d-1b7e-4436-b8eb-aa680a76f93d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:36.116660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:36.117715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:36.118146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:36.118603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] PATH_INFO: `/v3/projects/9df41e76392649f1aebd9c28476d1760/users/20f817e074ac4f00a1ad36994314d118/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:36.121814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df41e76392649f1aebd9c28476d1760, user_id=20f817e074ac4f00a1ad36994314d118, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:36.127883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:36.162991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 888/1774] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:36 2022] PUT /identity/v3/projects/9df41e76392649f1aebd9c28476d1760/users/20f817e074ac4f00a1ad36994314d118/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:36.165359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:36.166028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:36.166520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:36.219371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] MFA Rules not processed for user `20f817e074ac4f00a1ad36994314d118`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:36.220766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:36.351270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 887/1775] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:36.363526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:36.365478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:36.488624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e3d1c-258f-4686-acf9-e5a4cb060057 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:36.500618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:36.501647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:36.502070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:36.502662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:36.514816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:36.523322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:36.525077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:36.548986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 889/1776] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:36.593337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:36.782835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae7d37-d59c-421f-b787-66da76de37fc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:36.798572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:36.799591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:36.800033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:36.800448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] PATH_INFO: `/v3/users/c615664e3b2f49cdbd1ee341cf82b647` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:36.801948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c615664e3b2f49cdbd1ee341cf82b647)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:36.807964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:36.809632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:36.810745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:36.824832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:36.830338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a7ce41-fac2-4065-8053-9d4dce01742e service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:36.836441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:36.837236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:36.837640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:36.838375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] PATH_INFO: `/v3/projects/acb8cf4cfdc242ef81676957e2b81a56` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:36.841239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb8cf4cfdc242ef81676957e2b81a56)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:36.843420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:36.901863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c615664e3b2f49cdbd1ee341cf82b647'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.925101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c615664e3b2f49cdbd1ee341cf82b647'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.926464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c615664e3b2f49cdbd1ee341cf82b647'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.955881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb8cf4cfdc242ef81676957e2b81a56', 'request_id': 'req-7b8409de-b41a-40b2-a991-c9752de39d6a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.957815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c615664e3b2f49cdbd1ee341cf82b647'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.975717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb8cf4cfdc242ef81676957e2b81a56', 'request_id': 'req-7b8409de-b41a-40b2-a991-c9752de39d6a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.995958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb8cf4cfdc242ef81676957e2b81a56', 'request_id': 'req-7b8409de-b41a-40b2-a991-c9752de39d6a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:36.998268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e671986-0173-4b1d-9ef9-519fc3d331a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c615664e3b2f49cdbd1ee341cf82b647'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:37.023365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 888/1777] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:36 2022] DELETE /identity/v3/users/c615664e3b2f49cdbd1ee341cf82b647 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:37.028074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] The token cache is being invalidate because project acb8cf4cfdc242ef81676957e2b81a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:37.028376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:37.029045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e710a-28b6-497c-a133-97e131bd830c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.029335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e710a-28b6-497c-a133-97e131bd830c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.029620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e710a-28b6-497c-a133-97e131bd830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.030045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7b8409de-b41a-40b2-a991-c9752de39d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:37.032731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 890/1778] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:36 2022] DELETE /identity/v3/projects/acb8cf4cfdc242ef81676957e2b81a56 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:37.037478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.037984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.038399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.077908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-481e710a-28b6-497c-a133-97e131bd830c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:37.079151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-481e710a-28b6-497c-a133-97e131bd830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:37.094841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:37.110194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:37.226696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 889/1779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:37.300855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 891/1780] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:37.313692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.314019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.314324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.319063 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.328487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:37.370845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:37.377846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:37.402559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 890/1781] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:37.407349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.433639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371a2a9-82f8-49d2-878b-29d83aac4269 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.440615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.441396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.441644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.441934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.442738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:37.442862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.444621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:37.462074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 891/1782] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:37.465613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.488249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb553804-19b6-4715-81ea-d1b9e7f745ef None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.491006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb355b-8dfb-4b22-b65a-39856b6c6ff0 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.498206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.498513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.499233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.499521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.499797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.500150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.500852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.502076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.502652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.517478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.520441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:37.523270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:37.537804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 892/1783] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:37.580855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 892/1784] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:37 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:37.583432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.597491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d29878-f794-44ed-8a96-fd6aa0b83bcc service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.621796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:37.690563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.691365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.691631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.691998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.692941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.695074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:37.749182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 893/1785] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:37 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:37.749758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.766374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5e875-359a-48f8-b1d1-8a62b5580ec4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.783101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.783852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.784658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.785533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.786302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.788541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:37.818317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 893/1786] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:04:37.821189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.838444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976194c8-1f51-4ac0-9f61-820130b8d2a5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.852858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.854521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.856414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.856968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.858185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.864127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:37.892799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.894221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 894/1787] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:37.907495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e18486-26be-4b31-b15d-04db08a959f2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:37.924291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f37eb-25ec-4c76-bf42-74813d3e59e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:37.926419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f37eb-25ec-4c76-bf42-74813d3e59e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:37.927127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f37eb-25ec-4c76-bf42-74813d3e59e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:37.927620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f37eb-25ec-4c76-bf42-74813d3e59e8 admin admin] PATH_INFO: `/v3/projects/dccd0b5f434a45e78fd13eb76cac9a27/users/9cfd048f992343a88e245c6e30fcfe3d/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:37.974960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:37.977333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:37.988122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f37eb-25ec-4c76-bf42-74813d3e59e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccd0b5f434a45e78fd13eb76cac9a27, user_id=9cfd048f992343a88e245c6e30fcfe3d, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:37.991227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f37eb-25ec-4c76-bf42-74813d3e59e8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:38.025806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ba5-8718-4f0b-a97e-5b7888fc1df6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:38.046204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:38.047829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:38.048118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:38.049629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] PATH_INFO: `/v3/projects/333dddd51a5e4af89f8cfee915093ba3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:38.063823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=333dddd51a5e4af89f8cfee915093ba3)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:38.063915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 894/1788] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:37 2022] PUT /identity/v3/projects/dccd0b5f434a45e78fd13eb76cac9a27/users/9cfd048f992343a88e245c6e30fcfe3d/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:38.073556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:38.076335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea5e6-4f0e-484f-8ee4-9f51aaa19fa7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:38.076631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea5e6-4f0e-484f-8ee4-9f51aaa19fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:38.076948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faea5e6-4f0e-484f-8ee4-9f51aaa19fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:38.115801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-9faea5e6-4f0e-484f-8ee4-9f51aaa19fa7 None None] MFA Rules not processed for user `4b5e98fb43a143ea8cfa30990b4fffc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:38.117299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9faea5e6-4f0e-484f-8ee4-9f51aaa19fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:38.126284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:38.199347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333dddd51a5e4af89f8cfee915093ba3', 'request_id': 'req-1a1baba8-04c7-450d-810c-33a3258c6fdc'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.214046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333dddd51a5e4af89f8cfee915093ba3', 'request_id': 'req-1a1baba8-04c7-450d-810c-33a3258c6fdc'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.230088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333dddd51a5e4af89f8cfee915093ba3', 'request_id': 'req-1a1baba8-04c7-450d-810c-33a3258c6fdc'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.241634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 895/1789] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:38.243499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610a3dd-ca81-4271-976c-933178a27934 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:38.243793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610a3dd-ca81-4271-976c-933178a27934 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:38.245120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610a3dd-ca81-4271-976c-933178a27934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:38.255753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] The token cache is being invalidate because project 333dddd51a5e4af89f8cfee915093ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:38.256033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.257049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.260333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 895/1790] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:37 2022] DELETE /identity/v3/projects/333dddd51a5e4af89f8cfee915093ba3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:38.263941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:38.266292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:38.351918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4610a3dd-ca81-4271-976c-933178a27934 None None] MFA Rules not processed for user `9cfd048f992343a88e245c6e30fcfe3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:38.353743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4610a3dd-ca81-4271-976c-933178a27934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:38.383617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1baba8-04c7-450d-810c-33a3258c6fdc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:38.399748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:38.401614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:38.402093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:38.402379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] PATH_INFO: `/v3/users/20f817e074ac4f00a1ad36994314d118` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:38.403643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f817e074ac4f00a1ad36994314d118)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:38.405872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:38.438555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:38.515562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f817e074ac4f00a1ad36994314d118'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.532050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f817e074ac4f00a1ad36994314d118'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.533022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f817e074ac4f00a1ad36994314d118'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.551980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 896/1791] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:38.553814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:38.555639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:38.555639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:38.561491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f817e074ac4f00a1ad36994314d118'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.578576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f817e074ac4f00a1ad36994314d118'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:38.592790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 896/1792] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:38 2022] DELETE /identity/v3/users/20f817e074ac4f00a1ad36994314d118 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:38.597344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:38.601881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:38.621043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] MFA Rules not processed for user `1e3b1e0f4888400b89c47b3e44443228`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:38.622323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:38.755697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c25fe-34f2-4ded-8c04-0f982bd4a13b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:38.768661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:38.770628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:38.771432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:38.772380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:38.773355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:38.776095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:38.800906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 897/1793] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:38.868371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 897/1794] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:38 2022] POST /identity/v3/projects => generated 371 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:38.870373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:38.886896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b016c5f-51a3-4158-8bd4-b0ba58d8ec18 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:38.987259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:39.006817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.007704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.008038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.008321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.009232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.012517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.055678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 898/1795] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:38 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:39.057143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.086888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f67a99-0ed9-4744-9698-b60d289ba57e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.095410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.097130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:39.102146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.102782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.103064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.103408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.104071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.105884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.120103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367c8e4-e35a-4c1e-ae09-283863f55537 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.131402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.132227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.132517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.132830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] PATH_INFO: `/v3/projects/9df41e76392649f1aebd9c28476d1760` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.134311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df41e76392649f1aebd9c28476d1760)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.137153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.144775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 898/1796] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:39.149418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.163309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e2f76-693d-40e8-834a-c78c0db3eb2f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.170288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.171491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.172275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.172275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] PATH_INFO: `/v3/projects/38e57fbc11c74203b75b8e98be2e5925/users/648927cc23614a77897534adaafaa9d7/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.199744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e57fbc11c74203b75b8e98be2e5925, user_id=648927cc23614a77897534adaafaa9d7, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.204465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.211780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df41e76392649f1aebd9c28476d1760', 'request_id': 'req-86f0d927-4eab-4249-802c-30d23b5fd0d0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:39.225220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df41e76392649f1aebd9c28476d1760', 'request_id': 'req-86f0d927-4eab-4249-802c-30d23b5fd0d0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:39.228408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 899/1797] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:39 2022] PUT /identity/v3/projects/38e57fbc11c74203b75b8e98be2e5925/users/648927cc23614a77897534adaafaa9d7/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:39.230244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.257936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df41e76392649f1aebd9c28476d1760', 'request_id': 'req-86f0d927-4eab-4249-802c-30d23b5fd0d0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:39.278730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] The token cache is being invalidate because project 9df41e76392649f1aebd9c28476d1760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:39.279043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:39.280744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86f0d927-4eab-4249-802c-30d23b5fd0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:39.285881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 899/1798] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:39 2022] DELETE /identity/v3/projects/9df41e76392649f1aebd9c28476d1760 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:39.291016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5328c-1c23-4158-89fb-fc6856e5e41e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.291425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5328c-1c23-4158-89fb-fc6856e5e41e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.291709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5328c-1c23-4158-89fb-fc6856e5e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.308093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:39.330767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc066f-84ca-40de-8c5c-1ccca6f7bdc1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.335976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.336586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.336894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.337303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.338035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7fb5328c-1c23-4158-89fb-fc6856e5e41e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:39.338214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.339520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7fb5328c-1c23-4158-89fb-fc6856e5e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:39.339711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.354207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 900/1799] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:39.354393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 900/1800] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:39.355412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.357847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.358444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.358822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.359664 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:39.375277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146c281-588f-433a-934b-8f27ca49fed5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.382462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8da7a-6be7-46cd-9029-3b76313939df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.383545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:39.384079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8da7a-6be7-46cd-9029-3b76313939df admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.384599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8da7a-6be7-46cd-9029-3b76313939df admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.385427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:39.385599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8da7a-6be7-46cd-9029-3b76313939df admin admin] PATH_INFO: `/v3/projects/38e57fbc11c74203b75b8e98be2e5925/users/648927cc23614a77897534adaafaa9d7/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.387965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8da7a-6be7-46cd-9029-3b76313939df admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e57fbc11c74203b75b8e98be2e5925, user_id=648927cc23614a77897534adaafaa9d7, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.390831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8da7a-6be7-46cd-9029-3b76313939df admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.395667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 901/1801] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:39.399407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.416767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 901/1802] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:39 2022] PUT /identity/v3/projects/38e57fbc11c74203b75b8e98be2e5925/users/648927cc23614a77897534adaafaa9d7/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:39.416968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc5dde-99fd-4b56-a259-dbfca5c70033 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.420210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.420503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.420817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.481974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] MFA Rules not processed for user `648927cc23614a77897534adaafaa9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:39.488785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:39.516728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.517632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.518025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.518465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.519232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:39.521182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.521532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.543689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 902/1803] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:39.547845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.567602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafd45f-e6b7-4dea-84e2-9b43c052fa97 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.576174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.576839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.577374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.577749 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.578908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.581386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.605353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 902/1804] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:39.618711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.620426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:39.656439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 903/1805] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:39 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:39.658868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.754422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:39.772279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2f350-420a-4a0f-b055-8588d9841b72 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.850642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd38910-21a2-456a-8613-122b5ad7c2fe admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.864109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.865917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.866549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.867056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.868058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.872662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.912639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 904/1806] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:39 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:39.917190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:39.930544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.932088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.932645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.933089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.944068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfceebeb-c598-4289-a246-24c0ab3f9701 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:39.958848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:39.959484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:39.959801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:39.960087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:39.960993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:39.963684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:39.995276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 905/1807] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:39.998766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:40.003411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:40.018765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe66c43-551d-4c9c-9ced-8d9582d9e7df admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.038762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:40.039392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.039676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.040016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.040750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.042386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.083307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.083461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 906/1808] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:40.084382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:40.089154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.120849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 903/1809] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:04:40.122060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2783ff2-5606-4f9c-8355-98a3bc9430f9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.158233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23284b9d-0a29-4b2b-a0f5-7d2172dd49b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:40.158929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23284b9d-0a29-4b2b-a0f5-7d2172dd49b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.159205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23284b9d-0a29-4b2b-a0f5-7d2172dd49b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.161399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23284b9d-0a29-4b2b-a0f5-7d2172dd49b0 admin admin] PATH_INFO: `/v3/projects/65a9eb38972f4dbdb708cb08fb6eac3d/users/6480e153271345689ef680cc8a710b6c/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.250594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:40.251690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23284b9d-0a29-4b2b-a0f5-7d2172dd49b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a9eb38972f4dbdb708cb08fb6eac3d, user_id=6480e153271345689ef680cc8a710b6c, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.259697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23284b9d-0a29-4b2b-a0f5-7d2172dd49b0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.264422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e47be-e86d-4f79-9de4-ec9d92318c77 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.274048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:40.276265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.277258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.277789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.280199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:40.333525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 907/1810] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:40 2022] PUT /identity/v3/projects/65a9eb38972f4dbdb708cb08fb6eac3d/users/6480e153271345689ef680cc8a710b6c/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:40.335030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.335317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.335578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.378267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] MFA Rules not processed for user `6480e153271345689ef680cc8a710b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:40.379878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:40.520140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.522469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.538589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 904/1811] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:40.538987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 908/1812] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:40.546263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:40.566370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68eacc-9af6-4d61-a51c-3137dd6629b9 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.712031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:40.713824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:40.714145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:40.714454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:40.715691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:40.719495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:40.768084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 905/1813] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:40 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:40.771727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:40.799242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76f702-c28a-4cd8-bb65-39fa4c6d0f91 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:40.988491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:41.007059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:41.007995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.008435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.008878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.009699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:41.011894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:41.070772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 909/1814] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:40 2022] POST /identity/v3/users => generated 294 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:41.085636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:41.120605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba39f2-89cd-4d48-ae6f-f3cc44abea62 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:41.143027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:41.158584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.160622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.161358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.163680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:41.171586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:41.226307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 906/1815] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:41.227434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:41.257981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2654d4-5260-4866-b2e4-4df9e9008e42 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:41.278320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c7ee5-0269-4171-b966-8623b4fda2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:41.281592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c7ee5-0269-4171-b966-8623b4fda2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.284842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c7ee5-0269-4171-b966-8623b4fda2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.286406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c7ee5-0269-4171-b966-8623b4fda2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.287885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c7ee5-0269-4171-b966-8623b4fda2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:41.293007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c7ee5-0269-4171-b966-8623b4fda2d5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:41.347113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 910/1816] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:41.352384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:41.375010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214a8f-82e8-402a-8208-beeb76e31199 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:41.376635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:41.377284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.377563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.377835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] PATH_INFO: `/v3/projects/fe9a3171523f4248b4c84ae4cf2e6333/users/43d3963593e541b08fb616bcce7b25b1/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.467263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9a3171523f4248b4c84ae4cf2e6333, user_id=43d3963593e541b08fb616bcce7b25b1, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:41.480913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:41.530900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 907/1817] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:41 2022] PUT /identity/v3/projects/fe9a3171523f4248b4c84ae4cf2e6333/users/43d3963593e541b08fb616bcce7b25b1/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:41.542692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.543208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.543631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.646998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] MFA Rules not processed for user `43d3963593e541b08fb616bcce7b25b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:41.649111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:41.786554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:41.787028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 911/1818] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:41.840141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:41.971889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20917f-e17f-4a35-8ef4-55c1c370bed5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:41.980554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:41.981663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:41.981951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:41.982224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:41.983284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:41.986273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.045609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 908/1819] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:41 2022] POST /identity/v3/projects => generated 387 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:42.049567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.065328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3347e-2384-4410-a5e5-8b0e0b53d080 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:42.181161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:42.183905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.185770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.186659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.188432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:42.194207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.241698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 912/1820] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:42 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:42.246352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.262813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad54ac7-29a4-4d17-bbc9-24810ac45f07 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:42.272391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff7e47-1027-4410-a8e6-0b740fec0542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:42.273335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff7e47-1027-4410-a8e6-0b740fec0542 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.273754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff7e47-1027-4410-a8e6-0b740fec0542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.274131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff7e47-1027-4410-a8e6-0b740fec0542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.275014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff7e47-1027-4410-a8e6-0b740fec0542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:42.277304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff7e47-1027-4410-a8e6-0b740fec0542 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.340572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.349979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 909/1821] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:42.361029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719debe-2ce3-4968-8d70-ce47144d2e5b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:42.401615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:42.403605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.405173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.405462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] PATH_INFO: `/v3/projects/61d44cc59a4f434583d470be34d7f9f7/users/02aeadfb18304f42b39e4ff616731182/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.439510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.439860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.440202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.525944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d44cc59a4f434583d470be34d7f9f7, user_id=02aeadfb18304f42b39e4ff616731182, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:42.534911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.535967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:42.537622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:42.613905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 913/1822] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:42 2022] PUT /identity/v3/projects/61d44cc59a4f434583d470be34d7f9f7/users/02aeadfb18304f42b39e4ff616731182/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:42.615501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.676343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 910/1823] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:42.704137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:42.767853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2bdc9-920f-4e7d-a460-e8fc9ffde073 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:42.780626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:42.782241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.782732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.783206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.785315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:42.792625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.820502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 914/1824] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:42.827755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.852345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3243d7d-dd6a-4c15-a528-682ce79b1ff1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:42.864535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f427a33-84bc-47d6-b72b-9a602cf27653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:42.870688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f427a33-84bc-47d6-b72b-9a602cf27653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.871160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f427a33-84bc-47d6-b72b-9a602cf27653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.871588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f427a33-84bc-47d6-b72b-9a602cf27653 admin admin] PATH_INFO: `/v3/projects/61d44cc59a4f434583d470be34d7f9f7/users/02aeadfb18304f42b39e4ff616731182/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:42.880053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f427a33-84bc-47d6-b72b-9a602cf27653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d44cc59a4f434583d470be34d7f9f7, user_id=02aeadfb18304f42b39e4ff616731182, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:42.913189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f427a33-84bc-47d6-b72b-9a602cf27653 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:42.919604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:42.921216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:42.942837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 911/1825] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:42 2022] PUT /identity/v3/projects/61d44cc59a4f434583d470be34d7f9f7/users/02aeadfb18304f42b39e4ff616731182/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:42.954387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:42.954770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:42.955149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:43.106993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] MFA Rules not processed for user `02aeadfb18304f42b39e4ff616731182`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:43.127449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:43.278434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:43.305815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8eccc-5a3f-483f-975e-f0359a5561cb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:43.382362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 912/1826] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:43.398939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:43.402018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:43.449950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:43.450569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:43.450846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:43.451283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:43.614115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e761f-7954-4b45-a118-af7c1ca20abc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:43.624124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:43.625792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:43.627640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:43.628508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:43.647107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:43.661654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:43.666823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:43.680615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:43.686040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:43.707523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 913/1827] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:43 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:04:43.718086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 915/1828] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 799 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 17 21:04:45.896163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:45.916821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbfe28-1c1e-460f-8426-ad6c8d7b7018 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:45.955523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340b2e6-9f24-4e4b-bfb8-c02fdff1f81d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:45.956266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340b2e6-9f24-4e4b-bfb8-c02fdff1f81d service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:45.956551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340b2e6-9f24-4e4b-bfb8-c02fdff1f81d service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:45.963189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340b2e6-9f24-4e4b-bfb8-c02fdff1f81d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:46.113436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340b2e6-9f24-4e4b-bfb8-c02fdff1f81d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:46.115450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340b2e6-9f24-4e4b-bfb8-c02fdff1f81d service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:46.174543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 914/1829] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:46.320522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:46.322863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:46.534617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad3606-07c1-4a04-9b1f-d59a4819c0e9 service swift] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:46.561473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:46.562135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:46.562510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:46.562663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] PATH_INFO: `/v3/users/2e15b43261ca4481a3e6f27ab1ef1e38` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:46.567789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e15b43261ca4481a3e6f27ab1ef1e38)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:46.575521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:46.774189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e15b43261ca4481a3e6f27ab1ef1e38'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:46.793543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e15b43261ca4481a3e6f27ab1ef1e38'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:46.794479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e15b43261ca4481a3e6f27ab1ef1e38'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:46.808134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e15b43261ca4481a3e6f27ab1ef1e38'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:46.837825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e15b43261ca4481a3e6f27ab1ef1e38'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:46.851542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 916/1830] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:46 2022] DELETE /identity/v3/users/2e15b43261ca4481a3e6f27ab1ef1e38 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:46.854464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:46.855056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:46.855346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:46.927338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:46.928635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:47.058068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 915/1831] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:47.109175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.110465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:47.190706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092328c-85df-468b-9dd3-eb8fc99bab7c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.194514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.195127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.195408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.195683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.207149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.208749 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.221893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 917/1832] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:47.534779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.536390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:47.550084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00af3-2a91-438a-a6e9-09ec2eb2c49d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.555550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.556297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.556634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.556949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] PATH_INFO: `/v3/projects/2a8db2f14a6f4721aff71d14087c841b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.558155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8db2f14a6f4721aff71d14087c841b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.560762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.569599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:47.611804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8db2f14a6f4721aff71d14087c841b', 'request_id': 'req-e9c9cd30-c588-4435-92c2-8f8f910eb18d'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.618005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8db2f14a6f4721aff71d14087c841b', 'request_id': 'req-e9c9cd30-c588-4435-92c2-8f8f910eb18d'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.625025 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8db2f14a6f4721aff71d14087c841b', 'request_id': 'req-e9c9cd30-c588-4435-92c2-8f8f910eb18d'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.640596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] The token cache is being invalidate because project 2a8db2f14a6f4721aff71d14087c841b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:47.641030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.641657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.643131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 916/1833] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:47 2022] DELETE /identity/v3/projects/2a8db2f14a6f4721aff71d14087c841b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:47.647052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:47.657370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79416641-cd31-44c3-9c73-39db564645f8 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:47.665047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:47.717620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:47.718694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:47.718694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:47.718934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] PATH_INFO: `/v3/users/e9e36ab4a76843f48723e3cd69a9b8af` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:47.721740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9e36ab4a76843f48723e3cd69a9b8af)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:47.723903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:47.782522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9e36ab4a76843f48723e3cd69a9b8af'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.792547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9e36ab4a76843f48723e3cd69a9b8af'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.793734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9e36ab4a76843f48723e3cd69a9b8af'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.802521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9e36ab4a76843f48723e3cd69a9b8af'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.820264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-dd44dabe-35e0-4c38-b1ed-239c43f14fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9e36ab4a76843f48723e3cd69a9b8af'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:47.829558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 918/1834] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:47 2022] DELETE /identity/v3/users/e9e36ab4a76843f48723e3cd69a9b8af => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:48.182304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.182304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.200865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9cd30-c588-4435-92c2-8f8f910eb18d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.320809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.321370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.322409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.322568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] PATH_INFO: `/v3/projects/7272b73ad1ff43e4a0ee0881b4d40411` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.323835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7272b73ad1ff43e4a0ee0881b4d40411)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.325808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.382059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7272b73ad1ff43e4a0ee0881b4d40411', 'request_id': 'req-af45a01e-ea38-454f-8b3e-9e9ed2adc985'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:48.393108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7272b73ad1ff43e4a0ee0881b4d40411', 'request_id': 'req-af45a01e-ea38-454f-8b3e-9e9ed2adc985'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:48.401610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7272b73ad1ff43e4a0ee0881b4d40411', 'request_id': 'req-af45a01e-ea38-454f-8b3e-9e9ed2adc985'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:48.416445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] The token cache is being invalidate because project 7272b73ad1ff43e4a0ee0881b4d40411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:48.416911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:48.417404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-af45a01e-ea38-454f-8b3e-9e9ed2adc985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:48.419090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 917/1835] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:48 2022] DELETE /identity/v3/projects/7272b73ad1ff43e4a0ee0881b4d40411 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:48.424147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46264-68e6-4897-ab90-792453427b77 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.424465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46264-68e6-4897-ab90-792453427b77 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.424931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a46264-68e6-4897-ab90-792453427b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.445849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-04a46264-68e6-4897-ab90-792453427b77 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:48.447907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-04a46264-68e6-4897-ab90-792453427b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.520822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 919/1836] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:48.525029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.525406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.525732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.546312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:48.547632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:48.555577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 918/1837] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:48.560606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a46264-68e6-4897-ab90-792453427b77 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.570969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a46264-68e6-4897-ab90-792453427b77 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.576435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.577106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.577385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.577660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.578319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:48.578617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.580228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.592602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 920/1838] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:48.597267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.607899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefbdc1-5ca3-4001-9002-bdf346b961e9 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.612587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.613320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.613614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.613900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.614555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.616618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.663238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:48.700206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.708956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 919/1839] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:48.718274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc62ce8-c7e6-4ee5-bac0-d8840a899007 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.743151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:48.799675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.800434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.800700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.800994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.801645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.803636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.828341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 921/1840] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:48.832374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.843281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0bd02-5353-4c45-a595-36c39d3409f5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.848092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.848715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.848996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.849267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.849919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.851540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.865498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 920/1841] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:04:48.866764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.878223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc492f7-a5d9-4d18-ad45-ca64700d0d93 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.883042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d4693-8b87-4de6-9ee9-08cf80624028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.883663 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4693-8b87-4de6-9ee9-08cf80624028 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.883941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4693-8b87-4de6-9ee9-08cf80624028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.884960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d4693-8b87-4de6-9ee9-08cf80624028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.885878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d4693-8b87-4de6-9ee9-08cf80624028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.887517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d4693-8b87-4de6-9ee9-08cf80624028 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.899245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 922/1842] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:48.906500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:48.917499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942c75b-5507-4b37-abdd-df01e80e4bda admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:48.922796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:48.923546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.923892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.924250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] PATH_INFO: `/v3/projects/8450247f329d4a9a8a97642ca6e1f70b/users/e37a05010f234bc5875d2fd42f7ade3b/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:48.945366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8450247f329d4a9a8a97642ca6e1f70b, user_id=e37a05010f234bc5875d2fd42f7ade3b, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:48.948793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:48.964993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 921/1843] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:48 2022] PUT /identity/v3/projects/8450247f329d4a9a8a97642ca6e1f70b/users/e37a05010f234bc5875d2fd42f7ade3b/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:04:48.975409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:48.975648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:48.975923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.008616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] MFA Rules not processed for user `e37a05010f234bc5875d2fd42f7ade3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:49.009911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:49.101888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 923/1844] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:49.107538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.177916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4474a-3491-49cc-9819-cb6e91df9b35 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.182894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.183524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.183750 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.184021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.184654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.186581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.215144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 922/1845] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:49 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:49.219572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.230968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa2837-9863-492d-8541-042774ec0d2e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.305400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.306397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.306957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.307420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.308445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.310963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.339689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 924/1846] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:49 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:49.340267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.351705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf7d14-541e-412d-b9e7-9f00bcb46b8b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.357208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.357831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.358179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.358545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.359265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.360950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.372726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 923/1847] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:04:49.380037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.397303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999cf64-aaa1-4386-8d49-e6c454f7850c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.405120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.405822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.406119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.406380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] PATH_INFO: `/v3/projects/83fd0d85436f4da586203466ec82c112/users/e5b555e0d7404afcb3a93c05ff6c7d66/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.419223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:49.422551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fd0d85436f4da586203466ec82c112, user_id=e5b555e0d7404afcb3a93c05ff6c7d66, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.425650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.439866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 925/1848] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:49 2022] PUT /identity/v3/projects/83fd0d85436f4da586203466ec82c112/users/e5b555e0d7404afcb3a93c05ff6c7d66/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:49.443844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.453906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc2ce1-1f2b-43ec-b1d8-c989ad055262 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.513745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf2dfb-4d9d-456a-98d9-d92b10e3afae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.515575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2dfb-4d9d-456a-98d9-d92b10e3afae admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.515855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2dfb-4d9d-456a-98d9-d92b10e3afae admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.516134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf2dfb-4d9d-456a-98d9-d92b10e3afae admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.516818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf2dfb-4d9d-456a-98d9-d92b10e3afae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.518458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf2dfb-4d9d-456a-98d9-d92b10e3afae admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.529097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 924/1849] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:49.533044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.543854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91636e1a-7907-4e29-a6c0-ed036205becc admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.548387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.549108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.549392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.549666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] PATH_INFO: `/v3/projects/83fd0d85436f4da586203466ec82c112/users/e5b555e0d7404afcb3a93c05ff6c7d66/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.551540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fd0d85436f4da586203466ec82c112, user_id=e5b555e0d7404afcb3a93c05ff6c7d66, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.554201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.564696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 926/1850] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:49 2022] PUT /identity/v3/projects/83fd0d85436f4da586203466ec82c112/users/e5b555e0d7404afcb3a93c05ff6c7d66/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:49.575988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170248-ce38-4ece-817a-1283035cc43d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.576272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170248-ce38-4ece-817a-1283035cc43d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.576546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07170248-ce38-4ece-817a-1283035cc43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.606316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-07170248-ce38-4ece-817a-1283035cc43d None None] MFA Rules not processed for user `e5b555e0d7404afcb3a93c05ff6c7d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:49.607625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-07170248-ce38-4ece-817a-1283035cc43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:49.627509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:49.678432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 925/1851] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:49.690410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.692114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:49.749145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78afc1a2-7e80-44e3-a7de-d8cae83d2fcb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.752956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.753582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.753857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.754123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.763731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.765262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.774893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 927/1852] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:49.848952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170248-ce38-4ece-817a-1283035cc43d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:49.858756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07170248-ce38-4ece-817a-1283035cc43d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:49.862307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08553b70-a3ec-4ef1-a1cc-fc94b4e4dadc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:49.862901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08553b70-a3ec-4ef1-a1cc-fc94b4e4dadc service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:49.863179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08553b70-a3ec-4ef1-a1cc-fc94b4e4dadc service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:49.863448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08553b70-a3ec-4ef1-a1cc-fc94b4e4dadc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:49.923489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08553b70-a3ec-4ef1-a1cc-fc94b4e4dadc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:49.925001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08553b70-a3ec-4ef1-a1cc-fc94b4e4dadc service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:49.935377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 926/1853] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:52.298234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:52.299753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:52.384823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b9a6d-7999-4dd2-82c6-27ae8240c847 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:52.398027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:52.398683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:52.398970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:52.399265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] PATH_INFO: `/v3/users/3ee759071bb64f43a8019fa13b34ec6b` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:52.400957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee759071bb64f43a8019fa13b34ec6b)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:52.406892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:52.514971 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee759071bb64f43a8019fa13b34ec6b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:52.526928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee759071bb64f43a8019fa13b34ec6b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:52.530291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee759071bb64f43a8019fa13b34ec6b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:52.546096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee759071bb64f43a8019fa13b34ec6b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:52.564068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee759071bb64f43a8019fa13b34ec6b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:52.579461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:52.592063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 928/1854] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:52 2022] DELETE /identity/v3/users/3ee759071bb64f43a8019fa13b34ec6b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:52.596570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:52.597506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:52.597962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:52.642370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:52.646857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:52.766004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 927/1855] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:52.778196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:52.779964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:52.971760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc8362-cc3e-49e8-b7df-4a7f6bd45401 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:52.993307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:52.994181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:52.994401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:52.994784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:53.026270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:53.027851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:53.049352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 929/1856] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:53.371442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:53.373403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:53.387178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9f7bb-5504-42ae-a658-d12556235f31 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:53.399222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:53.416806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:53.417115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:53.417611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] PATH_INFO: `/v3/users/e37a05010f234bc5875d2fd42f7ade3b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:53.419129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37a05010f234bc5875d2fd42f7ade3b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:53.421287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:53.483681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:53.486533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:53.511257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43c48d-855b-4510-a29f-d28cf71d308e service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:53.541636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:53.542295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:53.542717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:53.542996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] PATH_INFO: `/v3/projects/73263f8118e54a469e1c7500e3539e21` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:53.546228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] RBAC: Authorizing `identity:delete_project(project_id=73263f8118e54a469e1c7500e3539e21)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:53.549155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:53.557143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37a05010f234bc5875d2fd42f7ade3b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.563221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:53.576920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37a05010f234bc5875d2fd42f7ade3b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.577693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37a05010f234bc5875d2fd42f7ade3b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.599239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37a05010f234bc5875d2fd42f7ade3b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.611200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f8ef64c-e68f-4fdb-9bb2-7933db437035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37a05010f234bc5875d2fd42f7ade3b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.642186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 928/1857] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:53 2022] DELETE /identity/v3/users/e37a05010f234bc5875d2fd42f7ade3b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:53.647312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:53.648487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:53.649117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:53.676543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73263f8118e54a469e1c7500e3539e21', 'request_id': 'req-6901f861-33d1-415b-84a1-f7653b2e492e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.686678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:53.687087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73263f8118e54a469e1c7500e3539e21', 'request_id': 'req-6901f861-33d1-415b-84a1-f7653b2e492e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.688190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:53.698088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73263f8118e54a469e1c7500e3539e21', 'request_id': 'req-6901f861-33d1-415b-84a1-f7653b2e492e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.711828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] The token cache is being invalidate because project 73263f8118e54a469e1c7500e3539e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:53.712614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.714122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.716803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 930/1858] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:53 2022] DELETE /identity/v3/projects/73263f8118e54a469e1c7500e3539e21 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:53.721416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:53.724238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:53.760269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 929/1859] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:53.771323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:53.773052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:53.805337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901f861-33d1-415b-84a1-f7653b2e492e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:53.813105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:53.814148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:53.814521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:53.814889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] PATH_INFO: `/v3/users/024ee409ec3b43a6b622ed4573cc3a72` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:53.816180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024ee409ec3b43a6b622ed4573cc3a72)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:53.818258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:53.856340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7fefd-3057-4888-b8e6-b0a8739b0b55 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:53.857187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:53.861392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:53.862124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:53.862407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:53.862736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:53.874667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:53.876430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:53.901605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024ee409ec3b43a6b622ed4573cc3a72'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.910330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 930/1860] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:53.912467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024ee409ec3b43a6b622ed4573cc3a72'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.913227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024ee409ec3b43a6b622ed4573cc3a72'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.921505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024ee409ec3b43a6b622ed4573cc3a72'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.934940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024ee409ec3b43a6b622ed4573cc3a72'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:53.940980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 931/1861] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:53 2022] DELETE /identity/v3/users/024ee409ec3b43a6b622ed4573cc3a72 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:54.340885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:54.347721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:54.416033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cd0d3-b7af-498b-8478-4fe034c06cf0 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:54.582378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:54.583018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:54.583298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:54.583573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] PATH_INFO: `/v3/projects/8450247f329d4a9a8a97642ca6e1f70b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:54.585071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8450247f329d4a9a8a97642ca6e1f70b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:54.590865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:54.613415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:54.669046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:54.671114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:54.705125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8450247f329d4a9a8a97642ca6e1f70b', 'request_id': 'req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:54.716978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8450247f329d4a9a8a97642ca6e1f70b', 'request_id': 'req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:54.737292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8450247f329d4a9a8a97642ca6e1f70b', 'request_id': 'req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:54.755800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] The token cache is being invalidate because project 8450247f329d4a9a8a97642ca6e1f70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:54.756268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:54.757434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:54.760607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 931/1862] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:54 2022] DELETE /identity/v3/projects/8450247f329d4a9a8a97642ca6e1f70b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:54.767270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:54.770283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:54.814198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b20177-3a80-4096-9e4e-83fa0ae62561 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:54.892643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98a964-f6e1-4afe-bcfe-2481b1ca0c2f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:54.903907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:54.905042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:54.905764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:54.906259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] PATH_INFO: `/v3/users/e5b555e0d7404afcb3a93c05ff6c7d66` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:54.907792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b555e0d7404afcb3a93c05ff6c7d66)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:54.911402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:54.949344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:54.950054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:54.950408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:54.951237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:54.953480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:55.008473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b555e0d7404afcb3a93c05ff6c7d66'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.033288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b555e0d7404afcb3a93c05ff6c7d66'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.034445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b555e0d7404afcb3a93c05ff6c7d66'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.084806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b555e0d7404afcb3a93c05ff6c7d66'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.094393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b555e0d7404afcb3a93c05ff6c7d66'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.096258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:55.098359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a537cc2-ce2e-47ea-872e-b30a31935671 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:55.107566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 932/1863] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:54 2022] DELETE /identity/v3/users/e5b555e0d7404afcb3a93c05ff6c7d66 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:55.114286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 932/1864] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:55.171024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:55.172706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:55.253123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:55.356651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3f3c9-b6ef-44eb-bd5e-7460a4445a13 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:55.367990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:55.371725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:55.372012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:55.373502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] PATH_INFO: `/v3/projects/c825198689d74640b9d7c4f1678af2c3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:55.375946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c825198689d74640b9d7c4f1678af2c3)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:55.382113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:55.504017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c825198689d74640b9d7c4f1678af2c3', 'request_id': 'req-bee12267-84fa-4768-b793-7961c717af5f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.517588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c825198689d74640b9d7c4f1678af2c3', 'request_id': 'req-bee12267-84fa-4768-b793-7961c717af5f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.527620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c825198689d74640b9d7c4f1678af2c3', 'request_id': 'req-bee12267-84fa-4768-b793-7961c717af5f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.553667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] The token cache is being invalidate because project c825198689d74640b9d7c4f1678af2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:55.554144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.554990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bee12267-84fa-4768-b793-7961c717af5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:55.558031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 933/1865] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:55 2022] DELETE /identity/v3/projects/c825198689d74640b9d7c4f1678af2c3 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:55.565038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:55.565444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:55.565751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:55.610504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:55.613873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:55.627327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:55.785502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 933/1866] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:55.800897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07c968-2f5b-4f7b-b3ff-af871618af18 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:55.801526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07c968-2f5b-4f7b-b3ff-af871618af18 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:55.801526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07c968-2f5b-4f7b-b3ff-af871618af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:55.869980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7c07c968-2f5b-4f7b-b3ff-af871618af18 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:55.880001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7c07c968-2f5b-4f7b-b3ff-af871618af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:55.911585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 934/1867] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:55.920448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:55.937029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a527d4-bffc-45df-809b-04dffc9dbd90 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:55.956880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:55.957613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:55.958096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:55.979116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:55.981267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:55.982469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:55.985786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:55.986726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:56.004551 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:56.013758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:56.088794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:56.106834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:56.117772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 934/1868] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:56.122028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:56.201171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffa7a5-c4e7-4c9e-8826-0d5326ab7c8b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:56.254059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:56.256650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:56.259611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:56.260129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:56.262897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:56.272108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:56.366801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 935/1869] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:56 2022] POST /identity/v3/projects => generated 387 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:56.369263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:56.373509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:56.419901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 935/1870] 10.222.0.62 () {58 vars in 1015 bytes} [Sat Sep 17 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:56.422604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:56.470755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4dce0f-8008-42f3-8719-784e73bd702e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:56.581931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007df56-5653-4263-bb99-db6c3886cd78 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:56.597460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:56.598353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:56.598636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:56.598910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] PATH_INFO: `/v3/projects/83fd0d85436f4da586203466ec82c112` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:56.601529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fd0d85436f4da586203466ec82c112)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:56.607247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:56.720356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:56.730677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fd0d85436f4da586203466ec82c112', 'request_id': 'req-0ed61752-4939-4d91-b4c8-f65867c3cfa4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:56.761206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fd0d85436f4da586203466ec82c112', 'request_id': 'req-0ed61752-4939-4d91-b4c8-f65867c3cfa4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:56.785868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fd0d85436f4da586203466ec82c112', 'request_id': 'req-0ed61752-4939-4d91-b4c8-f65867c3cfa4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:56.794983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:56.797298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:56.797746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:56.798100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:56.800117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:56.803479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] The token cache is being invalidate because project 83fd0d85436f4da586203466ec82c112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:04:56.803823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:56.804497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:56.811931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 936/1871] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:04:56 2022] DELETE /identity/v3/projects/83fd0d85436f4da586203466ec82c112 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:56.812509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:56.818974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:56.840230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:56.883079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:56.890781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 936/1872] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:56 2022] POST /identity/v3/users => generated 294 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:56.896730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:56.901583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.013326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed61752-4939-4d91-b4c8-f65867c3cfa4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.021225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.022248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.022791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.023395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.030304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.050174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.053547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.069621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8431c-4985-4288-9dd9-6d6be32d8c67 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.071058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 937/1873] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:57.072632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.074553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.079451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eebf55-8167-4597-b62e-a08411d92066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.082502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eebf55-8167-4597-b62e-a08411d92066 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.082794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eebf55-8167-4597-b62e-a08411d92066 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.083075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eebf55-8167-4597-b62e-a08411d92066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.092476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e839668-c776-4277-ba52-2259100892a1 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.097639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.098276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.098695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.099065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.109990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eebf55-8167-4597-b62e-a08411d92066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.111494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eebf55-8167-4597-b62e-a08411d92066 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.116382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.118990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.127031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 937/1874] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:57.128433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bbc94-18c9-40d8-b937-e446b98f7f85 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.133990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bbc94-18c9-40d8-b937-e446b98f7f85 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.134371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bbc94-18c9-40d8-b937-e446b98f7f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.156751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 938/1875] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:57.161069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.167361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.176654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e86bbc94-18c9-40d8-b937-e446b98f7f85 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:57.178018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e86bbc94-18c9-40d8-b937-e446b98f7f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.310639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1f75c-b713-4a1c-ac23-4d8cf65213be service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.319409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.321205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.322614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.323328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.324153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.326276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.333038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 938/1876] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:57.342172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.342456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.342841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.346856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 939/1877] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:57.351520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.377243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:57.378472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.398634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 939/1878] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:57.403648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.408260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030f298-0f8f-4aed-a22e-7e9d11545b6e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.419262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f676c2-bcdb-4028-9f87-f56c3ec129bd None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.434161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.435388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.436859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.436859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.436859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.437155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.437962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.437962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.438793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.439212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:04:57.441083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.442816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.443228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.457733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 940/1879] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:57.463339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.472462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 940/1880] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:04:57.476762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.477112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823baec-2c7c-4ae9-94e5-bddad4139dec admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.484285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739ad48-c3ac-486c-824a-13c818c60cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.485257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739ad48-c3ac-486c-824a-13c818c60cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.485919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739ad48-c3ac-486c-824a-13c818c60cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.486313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739ad48-c3ac-486c-824a-13c818c60cd5 admin admin] PATH_INFO: `/v3/projects/3dd9ed1f93894e41868cef493869e3c4/users/517fd44d2cf74b049ecbcbdad64c6f11/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.497553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58040f2-3e05-448b-b654-5f0d4339fb14 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.506682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.507480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.507897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.508567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.510043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.510644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739ad48-c3ac-486c-824a-13c818c60cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd9ed1f93894e41868cef493869e3c4, user_id=517fd44d2cf74b049ecbcbdad64c6f11, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.512314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.515903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739ad48-c3ac-486c-824a-13c818c60cd5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.555007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 941/1881] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:57 2022] PUT /identity/v3/projects/3dd9ed1f93894e41868cef493869e3c4/users/517fd44d2cf74b049ecbcbdad64c6f11/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:57.564440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 941/1882] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:57.569994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.570827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.570827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.580828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.630878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] MFA Rules not processed for user `517fd44d2cf74b049ecbcbdad64c6f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:57.632532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:57.677552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:57.701226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a8805-1026-45ac-8a60-4e231c348969 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.710848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.712326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.712326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.713953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.715225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.723034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.732854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:57.782813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 942/1883] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:57.785520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.787020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 942/1884] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:57.791843 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.817679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbad2d4-8de0-44b9-bbe2-462541ea6949 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.824080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec9727-d952-4ffb-b102-a8eea0a9b5cb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:57.831163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.832388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.835518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.836688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.836688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.839703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.840785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:57.841589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:57.842720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:57.843191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:57.846216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:57.848580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:57.869628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 943/1885] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:57.880763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.903412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 943/1886] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:57.905447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:57.907518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0498262-378e-4d94-8349-4e868a31e64d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.041591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.042768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.045796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.045796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.046454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.052080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.069435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f331f6-1fd6-4e0a-a974-d27be30604a6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.075547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 944/1887] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:58.077207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.085034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.085913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.086407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.087458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.090489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.097630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.116814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828193ff-bed1-4976-9636-5ee407df602f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.131985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91a746-d948-421b-bba1-eb6f86049009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.135211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91a746-d948-421b-bba1-eb6f86049009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.135701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91a746-d948-421b-bba1-eb6f86049009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.136256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91a746-d948-421b-bba1-eb6f86049009 admin admin] PATH_INFO: `/v3/projects/0fa6567f562842c189f7d3c96d232675/users/c26cd1261d904eaeba85ff5209518e9d/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.175074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91a746-d948-421b-bba1-eb6f86049009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa6567f562842c189f7d3c96d232675, user_id=c26cd1261d904eaeba85ff5209518e9d, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.183304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 944/1888] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:57 2022] POST /identity/v3/users => generated 299 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:58.186493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91a746-d948-421b-bba1-eb6f86049009 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.188023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.214845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 945/1889] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:58 2022] PUT /identity/v3/projects/0fa6567f562842c189f7d3c96d232675/users/c26cd1261d904eaeba85ff5209518e9d/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:58.225090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe47bad-b008-4cc5-905a-628951df80d8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.226354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.226721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.227005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.339701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] MFA Rules not processed for user `c26cd1261d904eaeba85ff5209518e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:58.345737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:58.403356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.404197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.404588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.405001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.405780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.410012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.451352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 945/1890] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:58.453414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.484363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e2f61-8a77-483b-a700-ccca35bf8884 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.516212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.518010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.519415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.519887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] PATH_INFO: `/v3/projects/19ce3a8f1f43416d955f368c7f554533/users/64474c85cbd1419f8eef08846a72429a/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.539094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 946/1891] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:58.540962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.580217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f3843-230a-4d47-be67-a281b0a6c51e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.588232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ce3a8f1f43416d955f368c7f554533, user_id=64474c85cbd1419f8eef08846a72429a, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.603992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.604618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.604917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.605188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.605830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.608410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.612606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.651520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 946/1892] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:58 2022] PUT /identity/v3/projects/19ce3a8f1f43416d955f368c7f554533/users/64474c85cbd1419f8eef08846a72429a/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:58.659190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.711593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 947/1893] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:04:58 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:04:58.715330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.721399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:58.737877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:58.779431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:58.833004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5438e77-c397-4a8d-b642-b0c52fae155f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.857626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7175849-e5e8-4315-9d37-53e372f259bc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.863360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.864007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.864304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.864630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] PATH_INFO: `/v3/users/6480e153271345689ef680cc8a710b6c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.866104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6480e153271345689ef680cc8a710b6c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.869708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.940724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6480e153271345689ef680cc8a710b6c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:58.947818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.948421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.948818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:58.948975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:58.949680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:58.953298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6480e153271345689ef680cc8a710b6c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:58.953984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:58.955876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6480e153271345689ef680cc8a710b6c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:58.970766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 947/1894] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:58.971705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:58.974370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:58.975522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6480e153271345689ef680cc8a710b6c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:58.987918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6480e153271345689ef680cc8a710b6c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:04:58.989208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f24e-8808-4d97-9c1a-fd55a1ae463f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:58.998535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499cd546-b484-45a4-a23d-7e1b56a36cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:58.999203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cd546-b484-45a4-a23d-7e1b56a36cca admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:58.999485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cd546-b484-45a4-a23d-7e1b56a36cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.000160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 948/1895] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:04:58 2022] DELETE /identity/v3/users/6480e153271345689ef680cc8a710b6c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:59.001250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.002839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:59.004206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499cd546-b484-45a4-a23d-7e1b56a36cca admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.005358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cd546-b484-45a4-a23d-7e1b56a36cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.007369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499cd546-b484-45a4-a23d-7e1b56a36cca admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.044560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 948/1896] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:04:58 2022] POST /identity/v3/users => generated 295 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:04:59.046690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.046995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.047368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.152205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:59.153529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:59.176780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d24ea-2790-4145-abbb-e542b7b7ad39 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.181509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 949/1897] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:04:59.182615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.191583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1441cf-4c06-4727-8a98-f47d8b84553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.194306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1441cf-4c06-4727-8a98-f47d8b84553f admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.197650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1441cf-4c06-4727-8a98-f47d8b84553f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.199344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1441cf-4c06-4727-8a98-f47d8b84553f admin admin] PATH_INFO: `/v3/projects/19ce3a8f1f43416d955f368c7f554533/users/64474c85cbd1419f8eef08846a72429a/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.204499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1441cf-4c06-4727-8a98-f47d8b84553f admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ce3a8f1f43416d955f368c7f554533, user_id=64474c85cbd1419f8eef08846a72429a, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.207025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e0ebf-7571-43d1-8fce-ed5bbd988c6e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.211907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1441cf-4c06-4727-8a98-f47d8b84553f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.218543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.219179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.219459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.219893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.220557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.224629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.241645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 949/1898] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:59 2022] PUT /identity/v3/projects/19ce3a8f1f43416d955f368c7f554533/users/64474c85cbd1419f8eef08846a72429a/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:59.248180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 950/1899] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:59.251860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.252332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.252619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.256304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.282093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2320f-7151-45c7-be7c-7cf6f0d27aa4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.322587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] MFA Rules not processed for user `64474c85cbd1419f8eef08846a72429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:59.331827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:59.343942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:59.405503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.406267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.406614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.406954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] PATH_INFO: `/v3/projects/cf0f97ecb8294f819c797bde0325b1bc/users/9d897c33e85a4a8fb791d1a1050cf007/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.427555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f97ecb8294f819c797bde0325b1bc, user_id=9d897c33e85a4a8fb791d1a1050cf007, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.431318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.454639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 951/1900] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:59 2022] PUT /identity/v3/projects/cf0f97ecb8294f819c797bde0325b1bc/users/9d897c33e85a4a8fb791d1a1050cf007/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:59.455648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.457511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:59.486928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 950/1901] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:04:59.486928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.502900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd802d13-a590-4ba3-a098-47ef67dee45d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.505797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:59.551304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2a35d-75cc-4195-87f2-47518fc212a5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.557331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.557957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.558238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.558513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.636430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.637311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.637762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.638185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.639171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.640963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.652751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.653879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 951/1902] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:04:59.654890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.655131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.657199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:59.678324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 952/1903] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:04:59.684339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:04:59.703120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa75a2-3c07-47ef-8d9f-17d46a65a9da service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.711628 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79bf2c-00ba-4f76-95a3-6a40019accf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.714122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79bf2c-00ba-4f76-95a3-6a40019accf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.714619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79bf2c-00ba-4f76-95a3-6a40019accf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.715043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79bf2c-00ba-4f76-95a3-6a40019accf8 admin admin] PATH_INFO: `/v3/projects/cf0f97ecb8294f819c797bde0325b1bc/users/9d897c33e85a4a8fb791d1a1050cf007/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.718408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79bf2c-00ba-4f76-95a3-6a40019accf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f97ecb8294f819c797bde0325b1bc, user_id=9d897c33e85a4a8fb791d1a1050cf007, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:04:59.722111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79bf2c-00ba-4f76-95a3-6a40019accf8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:04:59.740031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 953/1904] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:04:59 2022] PUT /identity/v3/projects/cf0f97ecb8294f819c797bde0325b1bc/users/9d897c33e85a4a8fb791d1a1050cf007/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:04:59.745578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaa327-ed7d-4b5d-b18f-b2ec9752223d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:04:59.751714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e89bc7-ba43-4bb0-a01a-e7d16c5db5f6 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.752530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e89bc7-ba43-4bb0-a01a-e7d16c5db5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.753508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e89bc7-ba43-4bb0-a01a-e7d16c5db5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.827067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-f8e89bc7-ba43-4bb0-a01a-e7d16c5db5f6 None None] MFA Rules not processed for user `9d897c33e85a4a8fb791d1a1050cf007`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:04:59.828718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f8e89bc7-ba43-4bb0-a01a-e7d16c5db5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:04:59.838001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd8e60-9c0d-4366-944c-efe1a60f003f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:04:59.839144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd8e60-9c0d-4366-944c-efe1a60f003f service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:04:59.840701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd8e60-9c0d-4366-944c-efe1a60f003f service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:04:59.841324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd8e60-9c0d-4366-944c-efe1a60f003f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:04:59.875272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:00.021525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 954/1905] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:00.021525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5967cb1-1eed-4581-b9a2-439d9279d092 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.021525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5967cb1-1eed-4581-b9a2-439d9279d092 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.023381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5967cb1-1eed-4581-b9a2-439d9279d092 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.025801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 955/1906] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:05:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:05:00.034943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd8e60-9c0d-4366-944c-efe1a60f003f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:00.036834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd8e60-9c0d-4366-944c-efe1a60f003f service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:00.049237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 952/1907] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:00.137991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5967cb1-1eed-4581-b9a2-439d9279d092 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:00.159086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5967cb1-1eed-4581-b9a2-439d9279d092 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:00.184538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:00.185258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.188861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.189253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.198022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:00.282760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:00.287766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.288086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.288396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.317243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:00.319930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3dc668-0793-474a-8383-c99f5df7fb26 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:00.332503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:00.336047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:00.337571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 956/1908] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:00.339944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e29b2c-e534-4905-b0f4-df8176e12a2a None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.340611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e29b2c-e534-4905-b0f4-df8176e12a2a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.341119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e29b2c-e534-4905-b0f4-df8176e12a2a None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.358185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 957/1909] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:05:00 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:05:00.452932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882c18b-69aa-43f2-a866-b900297028a5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.453687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882c18b-69aa-43f2-a866-b900297028a5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.454002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882c18b-69aa-43f2-a866-b900297028a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.513099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b882c18b-69aa-43f2-a866-b900297028a5 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:00.516221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 953/1910] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:00.517215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:00.519625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b882c18b-69aa-43f2-a866-b900297028a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:00.548923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 958/1911] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:00.676276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983d1da-dcf7-44cf-b9ce-caa073ddc6dc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:00.682435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:00.683134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.683473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.684478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] PATH_INFO: `/v3/users/517fd44d2cf74b049ecbcbdad64c6f11` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.685827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=517fd44d2cf74b049ecbcbdad64c6f11)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:00.687877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:00.779997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:00.799102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517fd44d2cf74b049ecbcbdad64c6f11'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:00.814258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517fd44d2cf74b049ecbcbdad64c6f11'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:00.815297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517fd44d2cf74b049ecbcbdad64c6f11'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:00.844404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517fd44d2cf74b049ecbcbdad64c6f11'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:00.855215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517fd44d2cf74b049ecbcbdad64c6f11'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:00.865276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 954/1912] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:00 2022] DELETE /identity/v3/users/517fd44d2cf74b049ecbcbdad64c6f11 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:00.870270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:00.870557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:00.886295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:00.906332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:00.907564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.016140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 959/1913] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:01.083423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:01.085702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.093305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:01.096125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.186777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8df0f6-84bd-402d-bf47-4d0187cbff07 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:01.193216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:01.193975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:01.197379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:01.197379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:01.209719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:01.211547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:01.223317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73482e-4f01-45c2-a25f-c4d0c2b81886 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:01.225438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 960/1914] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:01.230649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:01.231269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:01.231545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:01.232246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:01.233929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.252351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:01.266902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:01.270517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 955/1915] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:01.293871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:01.295742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.322359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda702c-9ff1-4a11-9473-43d3594cd528 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:01.329753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:01.330451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:01.330803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:01.331378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:01.333863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:01.356178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:01.361424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:01.384182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 961/1916] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:02.187146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:02.191971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:02.240868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419227e-2b5c-4655-a5f0-accbf0148a5a service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:02.248316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:02.249151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:02.249355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:02.249628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] PATH_INFO: `/v3/projects/65a9eb38972f4dbdb708cb08fb6eac3d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:02.251003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a9eb38972f4dbdb708cb08fb6eac3d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:02.253499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:02.343084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a9eb38972f4dbdb708cb08fb6eac3d', 'request_id': 'req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.350964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a9eb38972f4dbdb708cb08fb6eac3d', 'request_id': 'req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.359614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a9eb38972f4dbdb708cb08fb6eac3d', 'request_id': 'req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.384307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] The token cache is being invalidate because project 65a9eb38972f4dbdb708cb08fb6eac3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:02.385731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.386405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.396060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 956/1917] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:02 2022] DELETE /identity/v3/projects/65a9eb38972f4dbdb708cb08fb6eac3d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:02.400196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:02.420094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:02.421694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:02.425910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77427ad-3cfe-41f5-ade4-ebaec4c2c0a4 service nova] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:02.458199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:02.519058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb243a59-b28e-4f62-90ea-0ab9000fa3d2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:02.525824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:02.526203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:02.526672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:02.527182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] PATH_INFO: `/v3/projects/3dd9ed1f93894e41868cef493869e3c4` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:02.528415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd9ed1f93894e41868cef493869e3c4)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:02.530476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:02.538068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:02.538826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:02.539265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:02.539661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] PATH_INFO: `/v3/users/43d3963593e541b08fb616bcce7b25b1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:02.541248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d3963593e541b08fb616bcce7b25b1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:02.543400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:02.549635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:02.586398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd9ed1f93894e41868cef493869e3c4', 'request_id': 'req-fedff74c-034f-41f8-8227-6bc526b2bba6'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.593690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd9ed1f93894e41868cef493869e3c4', 'request_id': 'req-fedff74c-034f-41f8-8227-6bc526b2bba6'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.596321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d3963593e541b08fb616bcce7b25b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.600896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd9ed1f93894e41868cef493869e3c4', 'request_id': 'req-fedff74c-034f-41f8-8227-6bc526b2bba6'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.605129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d3963593e541b08fb616bcce7b25b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.605832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d3963593e541b08fb616bcce7b25b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.613238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d3963593e541b08fb616bcce7b25b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.614752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] The token cache is being invalidate because project 3dd9ed1f93894e41868cef493869e3c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:02.615128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.615771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.617314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 957/1918] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:02 2022] DELETE /identity/v3/projects/3dd9ed1f93894e41868cef493869e3c4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:02.621973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:02.623485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:02.626267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d3963593e541b08fb616bcce7b25b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.632314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 962/1919] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:02 2022] DELETE /identity/v3/users/43d3963593e541b08fb616bcce7b25b1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:02.688042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedff74c-034f-41f8-8227-6bc526b2bba6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:02.748172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:02.748829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:02.749103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:02.749383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] PATH_INFO: `/v3/users/64474c85cbd1419f8eef08846a72429a` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:02.750494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64474c85cbd1419f8eef08846a72429a)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:02.752418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:02.754928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:02.810427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64474c85cbd1419f8eef08846a72429a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.818682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64474c85cbd1419f8eef08846a72429a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.819434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64474c85cbd1419f8eef08846a72429a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.831864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64474c85cbd1419f8eef08846a72429a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.839280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64474c85cbd1419f8eef08846a72429a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:02.845520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 958/1920] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:02 2022] DELETE /identity/v3/users/64474c85cbd1419f8eef08846a72429a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:02.929951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:02.932320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:02.947340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e08ca1-a50d-4cd1-9fba-b7d13878bab2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.034047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.034657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.034920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.035179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] PATH_INFO: `/v3/projects/fe9a3171523f4248b4c84ae4cf2e6333` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.036361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9a3171523f4248b4c84ae4cf2e6333)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.038416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.089904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9a3171523f4248b4c84ae4cf2e6333', 'request_id': 'req-19626d45-03c3-4d63-96b3-05d6718ba9e2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.096715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9a3171523f4248b4c84ae4cf2e6333', 'request_id': 'req-19626d45-03c3-4d63-96b3-05d6718ba9e2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.105572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9a3171523f4248b4c84ae4cf2e6333', 'request_id': 'req-19626d45-03c3-4d63-96b3-05d6718ba9e2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.118894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] The token cache is being invalidate because project fe9a3171523f4248b4c84ae4cf2e6333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:03.119219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.119915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.122429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 963/1921] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:02 2022] DELETE /identity/v3/projects/fe9a3171523f4248b4c84ae4cf2e6333 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:03.126452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.128173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.146095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793fb24-9131-49a4-9351-cdc7a1f8fa4b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.210441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.211236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.211389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.211752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] PATH_INFO: `/v3/users/02aeadfb18304f42b39e4ff616731182` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.213201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=02aeadfb18304f42b39e4ff616731182)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.215156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.236130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.237692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.254419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19626d45-03c3-4d63-96b3-05d6718ba9e2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.260882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.261636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.261922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.262191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] PATH_INFO: `/v3/projects/19ce3a8f1f43416d955f368c7f554533` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.263558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ce3a8f1f43416d955f368c7f554533)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.265494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.284820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02aeadfb18304f42b39e4ff616731182'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.305041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:03.307484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02aeadfb18304f42b39e4ff616731182'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.308353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02aeadfb18304f42b39e4ff616731182'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.328200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02aeadfb18304f42b39e4ff616731182'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.336334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-769b3d64-b30c-40b3-8ece-4b6f8db15aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02aeadfb18304f42b39e4ff616731182'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.338575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:03.344284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 959/1922] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:03 2022] DELETE /identity/v3/users/02aeadfb18304f42b39e4ff616731182 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:03.348764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ce3a8f1f43416d955f368c7f554533', 'request_id': 'req-f4388fc5-2bcc-411e-9fca-be420dd1c206'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.357002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ce3a8f1f43416d955f368c7f554533', 'request_id': 'req-f4388fc5-2bcc-411e-9fca-be420dd1c206'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.370192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ce3a8f1f43416d955f368c7f554533', 'request_id': 'req-f4388fc5-2bcc-411e-9fca-be420dd1c206'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.403531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] The token cache is being invalidate because project 19ce3a8f1f43416d955f368c7f554533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:03.404079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.405721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4388fc5-2bcc-411e-9fca-be420dd1c206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.412087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.412373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.412648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.416657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 964/1923] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:03 2022] DELETE /identity/v3/projects/19ce3a8f1f43416d955f368c7f554533 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:03.434516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:03.436096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.514619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 960/1924] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:03.523536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.523820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.524131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.545275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:03.546480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.556365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 965/1925] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:03.560314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.570137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21587ca-fe92-4a8a-b81e-b90664e8f368 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.577231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.578748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.579318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.579576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.580499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:03.580879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.582766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.599174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 961/1926] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:05:03.600769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.612058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d5b8d-0501-4b8b-b1ec-ac9beddbd552 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.622052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.622877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.623635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.624003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.624769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.626824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.655821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 966/1927] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:03 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:03.659885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.671443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5ec29-3090-4ab4-ac4c-fcd26ddd11b1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.744285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.744831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.745107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.745374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.746087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.748015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.770201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 962/1928] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:03 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:03.774811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.785983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e5a6f-f8a2-472b-b503-a8b7ea3a847c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.791867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.792501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.792824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.793334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.793964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.795656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.809888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 967/1929] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:03.810851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.832841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.834303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ecabc-bbed-4dbc-9144-6e990b37cd91 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.834589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.839122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.841242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.841617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.842069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.842979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.846228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.847825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb8ec-7caa-4f72-8d1d-99e2aba56263 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.853758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.854397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.854682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.854963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] PATH_INFO: `/v3/projects/61d44cc59a4f434583d470be34d7f9f7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.856239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d44cc59a4f434583d470be34d7f9f7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.858100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.861828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 963/1930] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:03.865525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:03.875979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33acfe5-7e0d-41f0-afd7-5779d9eb7f7f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:03.880525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a27750-9d76-464d-a3fb-e4a6914cc1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:03.881183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a27750-9d76-464d-a3fb-e4a6914cc1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.881458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a27750-9d76-464d-a3fb-e4a6914cc1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.881727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a27750-9d76-464d-a3fb-e4a6914cc1ff admin admin] PATH_INFO: `/v3/projects/1fc77f889ba144c5b57f92f0d4d63594/users/35c4660da90541a887b106e77eccb6c4/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.896851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a27750-9d76-464d-a3fb-e4a6914cc1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc77f889ba144c5b57f92f0d4d63594, user_id=35c4660da90541a887b106e77eccb6c4, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:03.900423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a27750-9d76-464d-a3fb-e4a6914cc1ff admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:03.909044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d44cc59a4f434583d470be34d7f9f7', 'request_id': 'req-998d5245-d4d3-49d1-abb2-ac6c51999150'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.913870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 964/1931] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:03 2022] PUT /identity/v3/projects/1fc77f889ba144c5b57f92f0d4d63594/users/35c4660da90541a887b106e77eccb6c4/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:03.916701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d44cc59a4f434583d470be34d7f9f7', 'request_id': 'req-998d5245-d4d3-49d1-abb2-ac6c51999150'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.924085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d44cc59a4f434583d470be34d7f9f7', 'request_id': 'req-998d5245-d4d3-49d1-abb2-ac6c51999150'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.925503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.925789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.926071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.935450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] The token cache is being invalidate because project 61d44cc59a4f434583d470be34d7f9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:03.935738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.936389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-998d5245-d4d3-49d1-abb2-ac6c51999150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:03.937983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 968/1932] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:03 2022] DELETE /identity/v3/projects/61d44cc59a4f434583d470be34d7f9f7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:03.943835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26932de-2eae-49b7-83d0-b05415c8aa25 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:03.944121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26932de-2eae-49b7-83d0-b05415c8aa25 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:03.944394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26932de-2eae-49b7-83d0-b05415c8aa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:03.955201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] MFA Rules not processed for user `35c4660da90541a887b106e77eccb6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:03.956414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.963738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e26932de-2eae-49b7-83d0-b05415c8aa25 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:03.964925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e26932de-2eae-49b7-83d0-b05415c8aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:03.988552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:04.006262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:04.025896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 965/1933] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:04.029056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 969/1934] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:04.030315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.031630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:04.038017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.038306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.038585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.046079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a3534-c98f-4c66-a853-b74981b3f01b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.050861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.051465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.051745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.052018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.052821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.054858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.059484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:04.060711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:04.068794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 970/1935] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:04.073612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.083187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f381053-a61a-4352-888d-44384c0cccc7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.088501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.089141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.089423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.089698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.093198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 966/1936] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:04.093355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:04.093355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.093355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.093938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.109068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 971/1937] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:04.110647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.122666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851bc2e-bb5d-447b-ade8-4b10bc47f8c5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.176640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fde8b-495d-41b4-adfc-777c765a121b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.181927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.182567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.182813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.183092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.183737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.185880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.194903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.196582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.197548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.197889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.198646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.200856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.207572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 967/1938] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:04.211535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.220953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f405d-cbfd-42d4-8c3b-db0d9ec6be01 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.225670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.226250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.226526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.226794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.227496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.229125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.229677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 972/1939] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/projects => generated 405 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:04.233380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.240703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 968/1940] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:04.244614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.254395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec6120-2faa-4fd5-8782-70a5572ee7f7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.299725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7c202-06da-455d-ab86-f3492f2a8396 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.304346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.304980 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.305780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.306298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.306694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.308598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.317832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.318488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.319097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.319097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] PATH_INFO: `/v3/projects/e858c68c6fe84dab851e0c1038eb4731/users/ff73ce0756f84d978befede3e51473d9/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.336115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e858c68c6fe84dab851e0c1038eb4731, user_id=ff73ce0756f84d978befede3e51473d9, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.341584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 973/1941] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:05:04.341741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.342929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.353666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 969/1942] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:04 2022] PUT /identity/v3/projects/e858c68c6fe84dab851e0c1038eb4731/users/ff73ce0756f84d978befede3e51473d9/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:04.354517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065d722-2c00-412a-ab80-21ccb72bef31 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.357770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.370771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:04.392415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:04.431900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.432546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.432904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.433642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.434354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.437647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.450178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 974/1943] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:04.454145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.465316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1eee4-c0c2-461b-ac55-6bac747fc495 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.467131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80ebfe-5038-4090-9d95-520c94108fb3 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.470948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.471564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.471843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.472119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.472796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.473448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.474419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.474656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.475128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.475612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.476452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.479013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.486054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 970/1944] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:04.490210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.493898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 975/1945] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:04.496719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.507362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2aa2c-8998-4a69-90a9-608324a570af admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.507683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc45a6d-ca28-4553-bfe2-72faa5dcee3c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.513473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d845e98-1383-435d-bddc-0aa9a86cee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.514130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d845e98-1383-435d-bddc-0aa9a86cee77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.514406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d845e98-1383-435d-bddc-0aa9a86cee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.514533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcb4ba-1047-4ba7-9737-592c49f4808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.514808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d845e98-1383-435d-bddc-0aa9a86cee77 admin admin] PATH_INFO: `/v3/projects/1e36e57bc7f0411a9a1722dc1adaf67d/users/40695ab3ad554127b44138f7b8a1898f/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.515360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb4ba-1047-4ba7-9737-592c49f4808e admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.515639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb4ba-1047-4ba7-9737-592c49f4808e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.515908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb4ba-1047-4ba7-9737-592c49f4808e admin admin] PATH_INFO: `/v3/projects/e858c68c6fe84dab851e0c1038eb4731/users/ff73ce0756f84d978befede3e51473d9/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.518937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb4ba-1047-4ba7-9737-592c49f4808e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e858c68c6fe84dab851e0c1038eb4731, user_id=ff73ce0756f84d978befede3e51473d9, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.521725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb4ba-1047-4ba7-9737-592c49f4808e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.532177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d845e98-1383-435d-bddc-0aa9a86cee77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e36e57bc7f0411a9a1722dc1adaf67d, user_id=40695ab3ad554127b44138f7b8a1898f, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.534597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 971/1946] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:04 2022] PUT /identity/v3/projects/e858c68c6fe84dab851e0c1038eb4731/users/ff73ce0756f84d978befede3e51473d9/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:04.535042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d845e98-1383-435d-bddc-0aa9a86cee77 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.545759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.546151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.546428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.553993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 976/1947] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:04 2022] PUT /identity/v3/projects/1e36e57bc7f0411a9a1722dc1adaf67d/users/40695ab3ad554127b44138f7b8a1898f/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:04.565750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.566167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.566586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.578495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] MFA Rules not processed for user `ff73ce0756f84d978befede3e51473d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:04.579686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:04.599577 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] MFA Rules not processed for user `40695ab3ad554127b44138f7b8a1898f`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:04.601158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:04.658696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 972/1948] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:04.672545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.674871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:04.686671 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 977/1949] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:04.691150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.736014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01befb8c-273c-465e-a98e-4b853762c221 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.740559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.741324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.741689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.741875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.751908 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.753394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.755672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727c935-ae6b-4cc3-b3f7-e80fbdc1dc5d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.762329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.762946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.763228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.763500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.764355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 973/1950] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:04.764820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.766832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.804827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 978/1951] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/projects => generated 409 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:05:04.812214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.824111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddc4cd-1c1c-4521-b315-d302835ce2e7 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.873608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:04.927884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.937958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.938577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.938861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.939140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.939775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.941772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.942894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:04.967277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 974/1952] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:04 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:04.971289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:04.981598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9856e-290b-47e8-9a0d-d86755c9ca03 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:04.987205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:04.987801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:04.988720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:04.988720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:04.989047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:04.990653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:04.999925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a51057-ee05-44ed-adab-1b9d55201b01 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.001706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 975/1953] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:05.005993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.006873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.007525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.007875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.008155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] PATH_INFO: `/v3/users/35c4660da90541a887b106e77eccb6c4` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.009768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c4660da90541a887b106e77eccb6c4)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.011923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.016111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069975f-b40d-4246-9f49-1a9ef67d2aa2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.020636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.021316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.021595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.021892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] PATH_INFO: `/v3/projects/ea316283d0f441a49fc6f13adacfa35c/users/905e5da5d6bb4089988dee5ab8d324f8/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.038327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea316283d0f441a49fc6f13adacfa35c, user_id=905e5da5d6bb4089988dee5ab8d324f8, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.041988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.056623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 976/1954] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:05 2022] PUT /identity/v3/projects/ea316283d0f441a49fc6f13adacfa35c/users/905e5da5d6bb4089988dee5ab8d324f8/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:05.060609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.063442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c4660da90541a887b106e77eccb6c4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.079427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c4660da90541a887b106e77eccb6c4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.080261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c4660da90541a887b106e77eccb6c4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.088813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c4660da90541a887b106e77eccb6c4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.101632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-19739c4f-925f-4dfb-80be-96d59e814ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c4660da90541a887b106e77eccb6c4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.108433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 979/1955] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:04 2022] DELETE /identity/v3/users/35c4660da90541a887b106e77eccb6c4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:05.113208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.113493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.114346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.131686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5d70c-bbc1-4274-b9d5-7fe35f285fa3 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.135075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:05.136320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.191690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.192390 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.192716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.192983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.193672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.195221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.202228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 980/1956] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:05.207638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 977/1957] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:05.213831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.215464 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.216412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.218181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.227707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b592683-c582-4cba-aedd-6b95d2f430fb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.232949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdb6b6-27e1-49ff-88e8-009e697a9cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.233617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdb6b6-27e1-49ff-88e8-009e697a9cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.233964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdb6b6-27e1-49ff-88e8-009e697a9cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.234242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdb6b6-27e1-49ff-88e8-009e697a9cfa admin admin] PATH_INFO: `/v3/projects/ea316283d0f441a49fc6f13adacfa35c/users/905e5da5d6bb4089988dee5ab8d324f8/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.236179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdb6b6-27e1-49ff-88e8-009e697a9cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea316283d0f441a49fc6f13adacfa35c, user_id=905e5da5d6bb4089988dee5ab8d324f8, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.238861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdb6b6-27e1-49ff-88e8-009e697a9cfa admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.242675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:05.250090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 981/1958] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:05 2022] PUT /identity/v3/projects/ea316283d0f441a49fc6f13adacfa35c/users/905e5da5d6bb4089988dee5ab8d324f8/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:05.261408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.261689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.261967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.283785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305315ea-f3f5-43a5-82b6-08804e02fca5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.290744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] MFA Rules not processed for user `905e5da5d6bb4089988dee5ab8d324f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:05.291958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.301834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:05.345643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.346509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.346895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.347176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.397089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 982/1959] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:05.409057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.410519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.418679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.420072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.431150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 978/1960] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:05.468326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6f880-a2c8-4861-b541-aab95a0ef7b1 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.471898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.472499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.472813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.473082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.482597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.484086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.494198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 983/1961] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:05.658434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.669685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884244ab-61ba-4b21-88f2-41ec171bf6ae service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.690025 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.690898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.691365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.691637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.702308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.731431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.732955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:05.748898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed2826-0dc2-413f-8d77-aa228b3c902a service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.754388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.755613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.756044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.756335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] PATH_INFO: `/v3/projects/1fc77f889ba144c5b57f92f0d4d63594` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.757535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc77f889ba144c5b57f92f0d4d63594)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.760120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.785968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.787549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:05.800747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 979/1962] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:05.816559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc77f889ba144c5b57f92f0d4d63594', 'request_id': 'req-3b815e43-3210-467e-a991-fb63f3801844'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.831102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc77f889ba144c5b57f92f0d4d63594', 'request_id': 'req-3b815e43-3210-467e-a991-fb63f3801844'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.840984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc77f889ba144c5b57f92f0d4d63594', 'request_id': 'req-3b815e43-3210-467e-a991-fb63f3801844'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.857470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] The token cache is being invalidate because project 1fc77f889ba144c5b57f92f0d4d63594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:05.857736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.858575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:05.860531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 984/1963] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:05 2022] DELETE /identity/v3/projects/1fc77f889ba144c5b57f92f0d4d63594 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:05.864550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:05.878431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7884e6-e1d7-4b6d-86d8-1b0afad13d15 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:05.908938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:05.969134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:05.969911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:05.970193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:05.970468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] PATH_INFO: `/v3/users/ff73ce0756f84d978befede3e51473d9` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:05.971613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff73ce0756f84d978befede3e51473d9)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:05.973618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:06.035665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff73ce0756f84d978befede3e51473d9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.045254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff73ce0756f84d978befede3e51473d9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.046260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff73ce0756f84d978befede3e51473d9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.063076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff73ce0756f84d978befede3e51473d9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.073960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff73ce0756f84d978befede3e51473d9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.085049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 980/1964] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:05 2022] DELETE /identity/v3/users/ff73ce0756f84d978befede3e51473d9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:06.403175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:06.403175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:06.411399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:06.413458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:06.418996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b815e43-3210-467e-a991-fb63f3801844 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:06.461446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:06.489352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb5113e-7cff-497b-9734-dbca1f81453c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:06.495543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:06.496156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.496433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.496719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.497177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:06.498004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.498294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.498571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] PATH_INFO: `/v3/projects/e858c68c6fe84dab851e0c1038eb4731` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.499234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:06.499698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e858c68c6fe84dab851e0c1038eb4731)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:06.502108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:06.561543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e858c68c6fe84dab851e0c1038eb4731', 'request_id': 'req-cbcb229a-7a75-4735-9177-3922f5fba9a9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.564555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:06.568449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:06.569445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e858c68c6fe84dab851e0c1038eb4731', 'request_id': 'req-cbcb229a-7a75-4735-9177-3922f5fba9a9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.570535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bff886-00f6-4278-90f0-5f8f62382932 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:06.579461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e858c68c6fe84dab851e0c1038eb4731', 'request_id': 'req-cbcb229a-7a75-4735-9177-3922f5fba9a9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.584852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 981/1965] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:06.591875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] The token cache is being invalidate because project e858c68c6fe84dab851e0c1038eb4731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:06.592238 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.593040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cbcb229a-7a75-4735-9177-3922f5fba9a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:06.598321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 985/1966] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:06 2022] DELETE /identity/v3/projects/e858c68c6fe84dab851e0c1038eb4731 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:06.600323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.600634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.600947 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.622027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:06.623227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:06.693566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 982/1967] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:06.702337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.703335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.703677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.725791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:06.727690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:06.738658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 986/1968] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:06.742965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:06.754058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce1a88-57aa-4ad6-8594-bf1b61b75b8a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:06.764394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:06.765045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.765843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.765843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.766507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:06.767116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:06.769766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:06.786904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 983/1969] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:06.790584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:06.800082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00929d91-afc2-475e-883d-ba1a334e555c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:06.806021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:06.806611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.806896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.807177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.807804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:06.809733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:06.840412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 987/1970] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:06 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:06.845473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:06.857195 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621301d-7428-463e-9662-59775df5f1b4 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:06.919959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:06.920580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.920893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.921170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.921815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:06.923752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:06.953891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 984/1971] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:06 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:06.963191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:06.978722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4312a3-953d-4a7e-8141-779b2ecab028 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:06.983611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:06.984210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:06.984483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:06.984807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:06.985446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:06.987096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.001273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 988/1972] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:07.002911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.013869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdaad1-7a82-4a39-ae8b-2f0053bb275b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.020498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89636a04-0842-4e5e-b096-96915dde8e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.021222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89636a04-0842-4e5e-b096-96915dde8e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.021498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89636a04-0842-4e5e-b096-96915dde8e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.021773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89636a04-0842-4e5e-b096-96915dde8e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.022433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89636a04-0842-4e5e-b096-96915dde8e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.024742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89636a04-0842-4e5e-b096-96915dde8e52 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.035618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 985/1973] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:07.040092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.050154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ac1d-4582-4195-9f05-279ce8bf5bb6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.054846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.055917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.056194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.056468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] PATH_INFO: `/v3/projects/59f4e8750f0045cfb7db654a7ba716d1/users/977991ae41a74ad6a84fd332a5cbfc58/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.071967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f4e8750f0045cfb7db654a7ba716d1, user_id=977991ae41a74ad6a84fd332a5cbfc58, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.074523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.088532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 989/1974] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:07 2022] PUT /identity/v3/projects/59f4e8750f0045cfb7db654a7ba716d1/users/977991ae41a74ad6a84fd332a5cbfc58/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:07.098898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.099179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.099475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.127802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] MFA Rules not processed for user `977991ae41a74ad6a84fd332a5cbfc58`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:07.129133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:07.179457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:07.210912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 986/1975] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:07.215972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.278560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:07.284489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f8623-d48d-44a5-823a-903b86e20ae1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.290230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.291142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.291299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.291574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.292206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.294505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.334571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 990/1976] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:07 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:07.338555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.354708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a289f-447e-4376-8038-b033209639a1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.441919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.442669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.443000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.443402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.444038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.446283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.484445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 987/1977] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:07 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:07.488426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.500939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f491-3695-4c58-a4cd-b3f1e7712c02 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.508972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.509759 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.510107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.510446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.511315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.513100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.524240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 991/1978] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:07.528466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.538338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95c997-04d7-4b6f-9e6e-c35f4a29f7ff admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.543688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.544328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.544602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.544913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] PATH_INFO: `/v3/projects/cf35e9ac20b942f3a3691b61d5a7856c/users/d2a3e12f63954395b754b64e17c9b101/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.559979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf35e9ac20b942f3a3691b61d5a7856c, user_id=d2a3e12f63954395b754b64e17c9b101, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.562685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.575670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 988/1979] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:07 2022] PUT /identity/v3/projects/cf35e9ac20b942f3a3691b61d5a7856c/users/d2a3e12f63954395b754b64e17c9b101/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:07.579618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.589600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa968d8b-5113-4eac-8dd4-e7a5d5ccdfa7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.645267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86aac-1013-4e7a-8854-78e443c42e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.645783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86aac-1013-4e7a-8854-78e443c42e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.646062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86aac-1013-4e7a-8854-78e443c42e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.646331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86aac-1013-4e7a-8854-78e443c42e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.646963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86aac-1013-4e7a-8854-78e443c42e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.648552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86aac-1013-4e7a-8854-78e443c42e47 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.658123 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 992/1980] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:07.662380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.673286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d3be9-0215-427b-9d97-3f5f5807d7d1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.677742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.678377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.678647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.678914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] PATH_INFO: `/v3/projects/cf35e9ac20b942f3a3691b61d5a7856c/users/d2a3e12f63954395b754b64e17c9b101/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.681585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf35e9ac20b942f3a3691b61d5a7856c, user_id=d2a3e12f63954395b754b64e17c9b101, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.684201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.694288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 989/1981] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:07 2022] PUT /identity/v3/projects/cf35e9ac20b942f3a3691b61d5a7856c/users/d2a3e12f63954395b754b64e17c9b101/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:07.705702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.705987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.706264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.734812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] MFA Rules not processed for user `d2a3e12f63954395b754b64e17c9b101`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:07.736001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:07.803294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 993/1982] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:07.814707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.816393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:07.872923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa6fc5-b5f4-4a3f-ab26-9082cbc973f1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:07.876651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:07.877270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:07.877544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:07.877815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:07.887370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:07.888829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:07.898099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 990/1983] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:07.986744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:07.988223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.000461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd923fd-7607-4015-8ee0-e2bae29a2b7a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:08.004219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e692b-59e8-40fe-96ed-bbeee043378e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:08.004842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e692b-59e8-40fe-96ed-bbeee043378e service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.005119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e692b-59e8-40fe-96ed-bbeee043378e service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.005387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e692b-59e8-40fe-96ed-bbeee043378e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.053112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:08.062492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e692b-59e8-40fe-96ed-bbeee043378e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:08.063961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e692b-59e8-40fe-96ed-bbeee043378e service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:08.073240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 994/1984] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:08.532450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:08.583970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:08.604579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c76cb-54e4-417c-a564-8a71e59df37a service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:08.609007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:08.609631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.609901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.610170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] PATH_INFO: `/v3/users/977991ae41a74ad6a84fd332a5cbfc58` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.611322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=977991ae41a74ad6a84fd332a5cbfc58)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:08.613284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:08.681334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '977991ae41a74ad6a84fd332a5cbfc58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.690045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '977991ae41a74ad6a84fd332a5cbfc58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.690779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '977991ae41a74ad6a84fd332a5cbfc58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.704831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '977991ae41a74ad6a84fd332a5cbfc58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.719509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '977991ae41a74ad6a84fd332a5cbfc58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:08.725509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 991/1985] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:08 2022] DELETE /identity/v3/users/977991ae41a74ad6a84fd332a5cbfc58 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:08.729898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.730355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.730441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.749509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:08.750688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.808034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 995/1986] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:08.817907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:08.819302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:08.875843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ed5fc-2804-4329-9954-27821f388562 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:08.879439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:08.880013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:08.880287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:08.880554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:08.889843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:08.891269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:08.901536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 992/1987] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:09.163625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.165184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.177977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d967887-fa1b-4d80-a7e4-62ed608f0b05 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.182988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.183636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.183916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.184191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] PATH_INFO: `/v3/projects/59f4e8750f0045cfb7db654a7ba716d1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.185442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f4e8750f0045cfb7db654a7ba716d1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.187369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.241977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f4e8750f0045cfb7db654a7ba716d1', 'request_id': 'req-788024f0-31fd-458f-bb34-2edbb8d08aa0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.249418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f4e8750f0045cfb7db654a7ba716d1', 'request_id': 'req-788024f0-31fd-458f-bb34-2edbb8d08aa0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.257753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f4e8750f0045cfb7db654a7ba716d1', 'request_id': 'req-788024f0-31fd-458f-bb34-2edbb8d08aa0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.269219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] The token cache is being invalidate because project 59f4e8750f0045cfb7db654a7ba716d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:09.269502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.270171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.271666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 996/1988] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:09 2022] DELETE /identity/v3/projects/59f4e8750f0045cfb7db654a7ba716d1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:09.276621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.286484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb630a3a-5326-4e6d-9481-a3e70b2d8719 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.342936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.343555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.343831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.344106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] PATH_INFO: `/v3/users/d2a3e12f63954395b754b64e17c9b101` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.345251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a3e12f63954395b754b64e17c9b101)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.347185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.353133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:09.392779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a3e12f63954395b754b64e17c9b101'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.400500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a3e12f63954395b754b64e17c9b101'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.401128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a3e12f63954395b754b64e17c9b101'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.412728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a3e12f63954395b754b64e17c9b101'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.420066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5f471284-27a2-4f41-ac34-29459b45000f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a3e12f63954395b754b64e17c9b101'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.425826 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 993/1989] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:09 2022] DELETE /identity/v3/users/d2a3e12f63954395b754b64e17c9b101 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:09.791497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:09.793594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:09.809544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788024f0-31fd-458f-bb34-2edbb8d08aa0 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:09.879214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:09.879864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.880145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.880418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] PATH_INFO: `/v3/projects/cf35e9ac20b942f3a3691b61d5a7856c` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.881637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf35e9ac20b942f3a3691b61d5a7856c)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:09.884492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:09.904470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:09.937823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf35e9ac20b942f3a3691b61d5a7856c', 'request_id': 'req-f2a7aa23-a650-4413-9246-844ddbfefaa7'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.944807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf35e9ac20b942f3a3691b61d5a7856c', 'request_id': 'req-f2a7aa23-a650-4413-9246-844ddbfefaa7'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.955480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf35e9ac20b942f3a3691b61d5a7856c', 'request_id': 'req-f2a7aa23-a650-4413-9246-844ddbfefaa7'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.968505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] The token cache is being invalidate because project cf35e9ac20b942f3a3691b61d5a7856c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:09.968822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.969479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f2a7aa23-a650-4413-9246-844ddbfefaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:09.970978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 997/1990] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:09 2022] DELETE /identity/v3/projects/cf35e9ac20b942f3a3691b61d5a7856c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:09.976008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:09.976496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:09.976584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:09.998940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:10.000102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.081490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 994/1991] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:10.090038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.090325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.090600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.110682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:10.111895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.120931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 998/1992] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:10.125177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.135259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05cd54-890e-4570-85c0-ee7b074278a8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.140808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.141784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.142067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.142460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.153843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:10.153843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.153843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.162178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 995/1993] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:10.166944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.176349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0f84-67d0-4f85-9f27-e73e43174986 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.181874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.182468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.182747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.183019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.183652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.185590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.214224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 999/1994] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:10 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:05:10.218614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.228864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e997c-9a66-40ab-9c93-e628feb91085 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.284995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.285512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.285785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.286051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.286695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.288611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.309969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 996/1995] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:10 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:10.314182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.325340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3141296-b034-4899-9afe-1ff5e13b1752 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.330069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.330683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.330955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.331225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.331851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.333478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.343237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1000/1996] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:10.347146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.353091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:10.359598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646ebb2-6552-4a86-8454-646dddac9a64 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.365113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fffdac-8939-49a2-8015-535d908a73cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.365703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fffdac-8939-49a2-8015-535d908a73cd admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.365975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fffdac-8939-49a2-8015-535d908a73cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.366244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fffdac-8939-49a2-8015-535d908a73cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.366883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fffdac-8939-49a2-8015-535d908a73cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.368581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fffdac-8939-49a2-8015-535d908a73cd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.379408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 997/1997] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:10.384168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.394522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1c7e9-6975-4d3d-96c4-d72c07e00703 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.398593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.399828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.400101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.400394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] PATH_INFO: `/v3/projects/0e340708f839465dbfc58b3a4671ee81/users/4ca878f0a3f74097bfdf0bfb2bcd93b3/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.414676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e340708f839465dbfc58b3a4671ee81, user_id=4ca878f0a3f74097bfdf0bfb2bcd93b3, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.417392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.429692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1001/1998] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:10 2022] PUT /identity/v3/projects/0e340708f839465dbfc58b3a4671ee81/users/4ca878f0a3f74097bfdf0bfb2bcd93b3/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:10.440519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.440849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.441126 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.470252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] MFA Rules not processed for user `4ca878f0a3f74097bfdf0bfb2bcd93b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:10.471478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.533393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 998/1999] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:10.544588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.546305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.603612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750a8ec-31ce-4be2-b6f0-b1f43dd3f55a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.607352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ce7ab-cbc4-4ae3-bc5f-d36e00263cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.607952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ce7ab-cbc4-4ae3-bc5f-d36e00263cbf service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.608228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ce7ab-cbc4-4ae3-bc5f-d36e00263cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.608498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ce7ab-cbc4-4ae3-bc5f-d36e00263cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.616729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ce7ab-cbc4-4ae3-bc5f-d36e00263cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.618233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ce7ab-cbc4-4ae3-bc5f-d36e00263cbf service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.627135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1002/2000] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:10.646384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.708006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf354c-8763-4814-b38c-acdcbe959244 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.713539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.714163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.714434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.714708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] PATH_INFO: `/v3/users/4ca878f0a3f74097bfdf0bfb2bcd93b3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.715838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca878f0a3f74097bfdf0bfb2bcd93b3)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:10.717794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:10.764762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca878f0a3f74097bfdf0bfb2bcd93b3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:10.774602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca878f0a3f74097bfdf0bfb2bcd93b3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:10.775810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca878f0a3f74097bfdf0bfb2bcd93b3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:10.790473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca878f0a3f74097bfdf0bfb2bcd93b3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:10.797612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca878f0a3f74097bfdf0bfb2bcd93b3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:10.800121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:10.812616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 999/2001] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:10 2022] DELETE /identity/v3/users/4ca878f0a3f74097bfdf0bfb2bcd93b3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:10.817073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.817525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.817878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:10.837741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:10.839061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.887528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:10.910713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1003/2002] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:10.920743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:10.922165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:10.983717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581af16-5e19-4d4a-9abf-2c54038804f9 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:10.987477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5a975-4490-44ab-a537-2162f7f57be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:10.988081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a975-4490-44ab-a537-2162f7f57be6 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:10.988358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a975-4490-44ab-a537-2162f7f57be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:10.988801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a975-4490-44ab-a537-2162f7f57be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.003875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a975-4490-44ab-a537-2162f7f57be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.005396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a975-4490-44ab-a537-2162f7f57be6 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.015482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1000/2003] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:11.489835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.492316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:11.513263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a80174-27b8-4fed-a78b-d8c3bdff5a8f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.519289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.521506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.521792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.522079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] PATH_INFO: `/v3/projects/0e340708f839465dbfc58b3a4671ee81` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.523205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e340708f839465dbfc58b3a4671ee81)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.526065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.574222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e340708f839465dbfc58b3a4671ee81', 'request_id': 'req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.580460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e340708f839465dbfc58b3a4671ee81', 'request_id': 'req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.588718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e340708f839465dbfc58b3a4671ee81', 'request_id': 'req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.600435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] The token cache is being invalidate because project 0e340708f839465dbfc58b3a4671ee81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:11.600760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.601404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ffd1a31b-bb18-4503-93d6-3bbf3fa6e6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:11.603586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1004/2004] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:11 2022] DELETE /identity/v3/projects/0e340708f839465dbfc58b3a4671ee81 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:11.609102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.609461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.609702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.629958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:11.631181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:11.689568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1001/2005] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:11.698496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.698780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.699054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.719412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:11.720595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:11.729219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1005/2006] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:11.733322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.742778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3448c1-44fa-4c89-80d7-dabe589ebcbc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.747393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.747974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.748244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.748512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.749186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:11.749481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.751097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.762453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1002/2007] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:11.766772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.776341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3905716-4226-4937-907d-f66d7b05c129 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.781261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.781857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.782135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.782409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.783036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.784995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.813022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1006/2008] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:11 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:11.817204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.827337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc537e-bf8d-4699-a831-22487ed4f282 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.882916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.884189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.884467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.884780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.885405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.887344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.910568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1003/2009] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:11 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:05:11.915684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.926710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad576d72-1b54-4f83-a73f-ab214e4cca70 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.931344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.931933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.932212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.932484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.933152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.934750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.944554 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1007/2010] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:11.949572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.959408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9546c39d-4977-4b57-9807-48030dcfad48 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.964516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34870219-7524-4517-aa82-414545fe2c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.965135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34870219-7524-4517-aa82-414545fe2c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.965412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34870219-7524-4517-aa82-414545fe2c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.965685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34870219-7524-4517-aa82-414545fe2c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:11.966314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34870219-7524-4517-aa82-414545fe2c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:11.967905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34870219-7524-4517-aa82-414545fe2c35 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:11.978318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1004/2011] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:11.982292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:11.991900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd6eeae-1abc-425a-b09f-9106d2b2d663 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:11.996698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:11.997371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:11.997649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:11.997924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] PATH_INFO: `/v3/projects/781752a0f0564e64a2d5463aeb13092b/users/dc4c37d86a074354ad264c18fe8f1980/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.015530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781752a0f0564e64a2d5463aeb13092b, user_id=dc4c37d86a074354ad264c18fe8f1980, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.018800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.033902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1008/2012] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:11 2022] PUT /identity/v3/projects/781752a0f0564e64a2d5463aeb13092b/users/dc4c37d86a074354ad264c18fe8f1980/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:12.044232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.044792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.044792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.064466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:12.075439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] MFA Rules not processed for user `dc4c37d86a074354ad264c18fe8f1980`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:12.076646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.158139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1005/2013] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:12.169974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:12.171707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.236991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe095e-0908-4a8d-bdb1-08d9de878da3 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:12.240068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32dc6a-f822-42ef-841b-969947140d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:12.240709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32dc6a-f822-42ef-841b-969947140d9f service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.240985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32dc6a-f822-42ef-841b-969947140d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.241255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32dc6a-f822-42ef-841b-969947140d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.250470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32dc6a-f822-42ef-841b-969947140d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.251891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32dc6a-f822-42ef-841b-969947140d9f service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.261303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1009/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:12.282358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:12.348597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073641b-7237-4f5b-bfb6-8ff1f36e5c83 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:12.353307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:12.353929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.354205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.354480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] PATH_INFO: `/v3/users/dc4c37d86a074354ad264c18fe8f1980` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.355622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4c37d86a074354ad264c18fe8f1980)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.357573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.404901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4c37d86a074354ad264c18fe8f1980'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.413137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4c37d86a074354ad264c18fe8f1980'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.413770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4c37d86a074354ad264c18fe8f1980'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.425845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4c37d86a074354ad264c18fe8f1980'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.433823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4c37d86a074354ad264c18fe8f1980'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:12.442359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1006/2015] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:12 2022] DELETE /identity/v3/users/dc4c37d86a074354ad264c18fe8f1980 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:12.449426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.449713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.449990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.463616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:12.470952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:12.472831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.540799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1010/2016] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:12.551091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:12.552481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.575734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:12.612968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27f18c5-857d-4fb7-a86c-9f88b76b9bb4 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:12.619813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1eddf-0cac-4399-a0fe-d2075dd9f8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:12.620431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1eddf-0cac-4399-a0fe-d2075dd9f8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.620836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1eddf-0cac-4399-a0fe-d2075dd9f8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.621006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1eddf-0cac-4399-a0fe-d2075dd9f8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.630897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1eddf-0cac-4399-a0fe-d2075dd9f8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.632334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1eddf-0cac-4399-a0fe-d2075dd9f8f0 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.648221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1007/2017] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:12.914165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:12.916114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:12.932427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5d35b-d607-46d4-aff0-679f81a01d63 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:12.941154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:12.941777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:12.942052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:12.942331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] PATH_INFO: `/v3/projects/781752a0f0564e64a2d5463aeb13092b` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:12.943529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=781752a0f0564e64a2d5463aeb13092b)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:12.945475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:12.994154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781752a0f0564e64a2d5463aeb13092b', 'request_id': 'req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.000369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781752a0f0564e64a2d5463aeb13092b', 'request_id': 'req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.007407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781752a0f0564e64a2d5463aeb13092b', 'request_id': 'req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.019433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] The token cache is being invalidate because project 781752a0f0564e64a2d5463aeb13092b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:13.019728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.020366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-aa8f929e-8479-4b40-b7ad-0f7c1eb3a2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:13.021874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1011/2018] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:12 2022] DELETE /identity/v3/projects/781752a0f0564e64a2d5463aeb13092b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:13.027171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.027459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.027738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.046806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:13.048007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.107309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1008/2019] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:13.115163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.115762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.116069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.139469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:13.140708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.148601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1012/2020] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:13.152824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.162465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9f2fe-25b0-40e4-aff8-2544520ef7e8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.167203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.167797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.168072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.168344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.169017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:13.169317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.170917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.182825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1009/2021] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:13.186992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.196875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c2cd-4477-4d5d-9402-2c064023af78 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.201609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.202209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.202488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.202763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.203389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.205316 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.232409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1013/2022] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:13.236451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.246239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e23e03-908b-426d-812e-ebdc88143d73 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.317672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.318338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.318649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.318952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.319606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.321976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.344034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1010/2023] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:13.347863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.358157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e85e5-5f2d-4f82-83b3-07dce1af83a8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.363332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.363929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.364206 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.364476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.365154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.366802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.381994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1014/2024] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:13.386742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.400660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed024abb-0d03-44e3-8aa8-e88c58e48bcd admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.405648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7429f66-d923-4d55-b0a9-8bca62319dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.406275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7429f66-d923-4d55-b0a9-8bca62319dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.406551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7429f66-d923-4d55-b0a9-8bca62319dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.406817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7429f66-d923-4d55-b0a9-8bca62319dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.407449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7429f66-d923-4d55-b0a9-8bca62319dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.409094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7429f66-d923-4d55-b0a9-8bca62319dd6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.420654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1011/2025] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:13.425635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.436872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a563b-cc9c-4bd8-9062-62c66a4c8740 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.441917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.442739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.443141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.443526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] PATH_INFO: `/v3/projects/d542e5d2c86d44758a8577e202abe81c/users/16955da09561483387b1245518607229/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.460432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d542e5d2c86d44758a8577e202abe81c, user_id=16955da09561483387b1245518607229, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.464967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.482323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1015/2026] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:13 2022] PUT /identity/v3/projects/d542e5d2c86d44758a8577e202abe81c/users/16955da09561483387b1245518607229/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:13.494812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06276e-36c5-4759-b32c-5b6b09f8014b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.495183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06276e-36c5-4759-b32c-5b6b09f8014b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.495525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06276e-36c5-4759-b32c-5b6b09f8014b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.524105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5d06276e-36c5-4759-b32c-5b6b09f8014b None None] MFA Rules not processed for user `16955da09561483387b1245518607229`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:13.525322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5d06276e-36c5-4759-b32c-5b6b09f8014b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.537379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:13.588151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1012/2027] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:13.599126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:13.600867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:13.631896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:13.686317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13671d-466c-4d4a-9bc1-0420e3fb912c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:13.690037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13415fd3-9e45-435b-a6f1-fb52ae758fad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:13.690754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415fd3-9e45-435b-a6f1-fb52ae758fad service glance] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:13.691043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415fd3-9e45-435b-a6f1-fb52ae758fad service glance] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:13.691319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13415fd3-9e45-435b-a6f1-fb52ae758fad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:13.699862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415fd3-9e45-435b-a6f1-fb52ae758fad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:13.701340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13415fd3-9e45-435b-a6f1-fb52ae758fad service glance] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:13.710138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1016/2028] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:20.491733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55a74-8e2a-46ec-99db-47d127ede74c None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:20.492524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55a74-8e2a-46ec-99db-47d127ede74c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:20.492524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd55a74-8e2a-46ec-99db-47d127ede74c None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:20.494328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1013/2029] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:05:20.509287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:20.509841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:20.509841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:20.538594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:20.540813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:20.618093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1017/2030] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:20.630156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd55a74-8e2a-46ec-99db-47d127ede74c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:20.631907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9bd55a74-8e2a-46ec-99db-47d127ede74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:20.650701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd55a74-8e2a-46ec-99db-47d127ede74c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:20.657261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127232b-b450-4851-98f0-a646923ea47b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:20.658009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127232b-b450-4851-98f0-a646923ea47b service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:20.658380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127232b-b450-4851-98f0-a646923ea47b service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:20.658675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127232b-b450-4851-98f0-a646923ea47b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:20.669143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127232b-b450-4851-98f0-a646923ea47b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:20.670778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127232b-b450-4851-98f0-a646923ea47b service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:20.681545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1014/2031] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:30.608760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=54 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:30.676974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=26 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:35.114203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:35.115751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:35.197839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665a824-19c3-4ef8-ada3-56248aea5a2e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:35.222611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:35.223286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:35.223811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:35.224100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] PATH_INFO: `/v3/users/40695ab3ad554127b44138f7b8a1898f` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:35.225281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] RBAC: Authorizing `identity:delete_user(user_id=40695ab3ad554127b44138f7b8a1898f)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:35.227698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:35.284862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40695ab3ad554127b44138f7b8a1898f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.292604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40695ab3ad554127b44138f7b8a1898f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.293309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40695ab3ad554127b44138f7b8a1898f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.302837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40695ab3ad554127b44138f7b8a1898f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.317825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40695ab3ad554127b44138f7b8a1898f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.329149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1018/2032] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:35 2022] DELETE /identity/v3/users/40695ab3ad554127b44138f7b8a1898f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:35.330636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:35.331111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:35.331221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:35.359071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:35.360292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:35.435657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1015/2033] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:35.446082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:35.447753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:35.503987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf267e4c-5478-488f-86f8-3e7a6580411f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:35.508353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:35.508997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:35.509277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:35.509549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:35.518553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:35.520076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:35.530302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1019/2034] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:35.860711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:35.862922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:35.877094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d93f3d-c620-4da3-beb3-b9642c0f7205 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:35.894625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:35.896233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:35.896772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:35.897206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] PATH_INFO: `/v3/projects/1e36e57bc7f0411a9a1722dc1adaf67d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:35.899208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e36e57bc7f0411a9a1722dc1adaf67d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:35.902587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:35.975599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e36e57bc7f0411a9a1722dc1adaf67d', 'request_id': 'req-60e4236d-6b50-4404-b494-22c75e828043'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.981245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e36e57bc7f0411a9a1722dc1adaf67d', 'request_id': 'req-60e4236d-6b50-4404-b494-22c75e828043'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:35.987510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e36e57bc7f0411a9a1722dc1adaf67d', 'request_id': 'req-60e4236d-6b50-4404-b494-22c75e828043'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.000169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] The token cache is being invalidate because project 1e36e57bc7f0411a9a1722dc1adaf67d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:36.000456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.001117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.002625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1016/2035] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:35 2022] DELETE /identity/v3/projects/1e36e57bc7f0411a9a1722dc1adaf67d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:36.006770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:36.017044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9498780-3d40-4fdb-86ee-9bdf657f933a service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:36.035837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:36.075306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:36.075931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:36.076215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:36.076491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] PATH_INFO: `/v3/users/905e5da5d6bb4089988dee5ab8d324f8` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:36.077629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] RBAC: Authorizing `identity:delete_user(user_id=905e5da5d6bb4089988dee5ab8d324f8)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:36.079531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:36.124839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905e5da5d6bb4089988dee5ab8d324f8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.131849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905e5da5d6bb4089988dee5ab8d324f8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.133017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905e5da5d6bb4089988dee5ab8d324f8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.141028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905e5da5d6bb4089988dee5ab8d324f8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.154343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e354a0dc-8c2b-4f46-9269-8e2fcd5c067e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905e5da5d6bb4089988dee5ab8d324f8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:36.160594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1020/2036] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:36 2022] DELETE /identity/v3/users/905e5da5d6bb4089988dee5ab8d324f8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:36.865389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:36.866925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:36.887670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4236d-6b50-4404-b494-22c75e828043 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.070693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.072258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.072552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.073411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] PATH_INFO: `/v3/projects/ea316283d0f441a49fc6f13adacfa35c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.074641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea316283d0f441a49fc6f13adacfa35c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.100072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.207302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:37.232323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea316283d0f441a49fc6f13adacfa35c', 'request_id': 'req-a362369f-84bc-4960-ab52-5e82deb22a02'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:37.239260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea316283d0f441a49fc6f13adacfa35c', 'request_id': 'req-a362369f-84bc-4960-ab52-5e82deb22a02'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:37.249248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea316283d0f441a49fc6f13adacfa35c', 'request_id': 'req-a362369f-84bc-4960-ab52-5e82deb22a02'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:37.265387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] The token cache is being invalidate because project ea316283d0f441a49fc6f13adacfa35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:37.265608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:37.266241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a362369f-84bc-4960-ab52-5e82deb22a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:37.268167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1017/2037] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:36 2022] DELETE /identity/v3/projects/ea316283d0f441a49fc6f13adacfa35c => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:37.274177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.274674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.274754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.305246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:37.306721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:37.443699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1021/2038] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:37.452506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.452818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.454960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.487817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:37.489838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:37.506095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1018/2039] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:37.508137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.520226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd92b-612f-4d2f-ac7f-28109ec2eece None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.525973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.526581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.526857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.527127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.527774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:05:37.528073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.529702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.543320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1022/2040] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:05:37.554359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.583572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04626f21-7058-4475-93d5-e06cb5848137 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.589196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.589893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.590347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.590891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.591824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.594282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.645506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1019/2041] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:37.650244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.663344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a679844-213e-4e5c-a657-5e01d5727ed1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.701078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:37.768152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.768883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.769236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.769582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.770282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.772276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.794909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1023/2042] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/users => generated 283 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:05:37.799087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.810274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b3c6e-93b2-4820-b1dd-08966f9e23d2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.814964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.815629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.815973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.816313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.817114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.818816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.831049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1020/2043] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:37.834903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.846691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33dff7d-adbc-48f6-9561-8a29e2e2c261 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.856338 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823306e-3d20-416b-828a-9e618d024cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.857007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823306e-3d20-416b-828a-9e618d024cba admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.857292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823306e-3d20-416b-828a-9e618d024cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.857603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823306e-3d20-416b-828a-9e618d024cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.858246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823306e-3d20-416b-828a-9e618d024cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.859852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823306e-3d20-416b-828a-9e618d024cba admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.874713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1024/2044] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:05:37.875801 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:37.886562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e33e4-2cf7-4b6c-ae08-3b4d49501e02 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:37.891359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:37.892010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.892291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.892571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] PATH_INFO: `/v3/projects/d71daf73ae764e8a9ccd07a31cbad6f7/users/f5bd158849d0465589432b33a2d1f927/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.907481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71daf73ae764e8a9ccd07a31cbad6f7, user_id=f5bd158849d0465589432b33a2d1f927, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:37.910234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:37.922742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1021/2045] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:37 2022] PUT /identity/v3/projects/d71daf73ae764e8a9ccd07a31cbad6f7/users/f5bd158849d0465589432b33a2d1f927/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:37.933973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:37.934274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:37.934556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:37.963177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] MFA Rules not processed for user `f5bd158849d0465589432b33a2d1f927`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:37.964396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.027895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1025/2046] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:38.032259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.094143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff4e4b-84e7-45c8-b5f0-58952afc6a71 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.098988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.099602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.099876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.100148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.101297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.103299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.129406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1022/2047] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:38.133505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.144272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ced74-e963-46ba-8778-47fe1be2d2d9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.199776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.200381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.200658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.200962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.201584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.203500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.223696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1026/2048] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:05:38.228469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.238501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4907e25-2b81-49e0-9cad-e5b36a652bdf admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.244262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.244903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.245180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.245455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.246087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.247685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.258518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1023/2049] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:38.262585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.272609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d154c6-3aa5-45f9-a0ee-382df5fb3a6b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.277427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.278142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.278362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.278631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] PATH_INFO: `/v3/projects/98d68a64542b4b33872afe9a8c8ce8ef/users/d5adfbc9159949dd93c8e1e406cc1bc7/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.293561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d68a64542b4b33872afe9a8c8ce8ef, user_id=d5adfbc9159949dd93c8e1e406cc1bc7, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.296197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.309990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1027/2050] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:38 2022] PUT /identity/v3/projects/98d68a64542b4b33872afe9a8c8ce8ef/users/d5adfbc9159949dd93c8e1e406cc1bc7/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:38.313840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.325415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be178ecd-91e0-49ab-aadd-2e5fc174834c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.345425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:38.384135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d251b-1e2f-4231-9893-3a021d4067dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.384805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d251b-1e2f-4231-9893-3a021d4067dd admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.385080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d251b-1e2f-4231-9893-3a021d4067dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.385373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d251b-1e2f-4231-9893-3a021d4067dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.386005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d251b-1e2f-4231-9893-3a021d4067dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.387599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d251b-1e2f-4231-9893-3a021d4067dd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.398596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1024/2051] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:05:38.402717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.408047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:38.415059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7fca5-888b-430b-9553-674edb3f13a6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.420190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.420878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.421159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.421431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] PATH_INFO: `/v3/projects/98d68a64542b4b33872afe9a8c8ce8ef/users/d5adfbc9159949dd93c8e1e406cc1bc7/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.423324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d68a64542b4b33872afe9a8c8ce8ef, user_id=d5adfbc9159949dd93c8e1e406cc1bc7, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.426077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.437147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1028/2052] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:05:38 2022] PUT /identity/v3/projects/98d68a64542b4b33872afe9a8c8ce8ef/users/d5adfbc9159949dd93c8e1e406cc1bc7/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:05:38.448527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.448989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.449308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.477363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] MFA Rules not processed for user `d5adfbc9159949dd93c8e1e406cc1bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:38.478553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.543765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1025/2053] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:38.555349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.557228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:38.626915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071dc0-5994-422a-8d60-fd042a147da5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.631241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.631848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.632172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.632442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.642030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.643473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.654052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1029/2054] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:38.750917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:38.762834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187075b2-314b-4e5e-a159-0f8d7e77cc50 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:38.766582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004195b-3566-4973-83d8-56551a8c9ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:38.768612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004195b-3566-4973-83d8-56551a8c9ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:38.768612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004195b-3566-4973-83d8-56551a8c9ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:38.768612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004195b-3566-4973-83d8-56551a8c9ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:38.836803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004195b-3566-4973-83d8-56551a8c9ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:38.838669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004195b-3566-4973-83d8-56551a8c9ce3 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:38.848464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1026/2055] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:05:48.649571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:48.844315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=80 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:58.572162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:58.640409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a8f26-f064-450b-a287-5e819b6dd1c9 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:58.645201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:58.645832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:58.646113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:58.646389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] PATH_INFO: `/v3/users/f5bd158849d0465589432b33a2d1f927` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:58.647680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5bd158849d0465589432b33a2d1f927)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:58.649766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:58.731389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5bd158849d0465589432b33a2d1f927'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:58.739394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5bd158849d0465589432b33a2d1f927'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:58.740061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5bd158849d0465589432b33a2d1f927'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:58.748286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5bd158849d0465589432b33a2d1f927'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:58.770641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5bd158849d0465589432b33a2d1f927'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:58.781752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1030/2056] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:58 2022] DELETE /identity/v3/users/f5bd158849d0465589432b33a2d1f927 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:58.786340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:58.786854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:58.786955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:58.813071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:58.814716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:58.886679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1027/2057] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:05:58.897788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:58.899282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:58.960035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4fe16-7074-4b07-a68c-86240fdc0cfc admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:58.963754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4a74b-c2a0-425b-be2e-d00d43205e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:58.964356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4a74b-c2a0-425b-be2e-d00d43205e8e service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:58.964631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4a74b-c2a0-425b-be2e-d00d43205e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:58.964933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4a74b-c2a0-425b-be2e-d00d43205e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:58.974201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4a74b-c2a0-425b-be2e-d00d43205e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:58.976117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4a74b-c2a0-425b-be2e-d00d43205e8e service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:58.986466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1031/2058] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:59.139385 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:59.141348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:59.160313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f1f3e-e183-4ec3-bce7-d425198ffafa None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:59.170459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:59.171301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:59.171791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:59.172064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] PATH_INFO: `/v3/users/16955da09561483387b1245518607229` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:59.173443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16955da09561483387b1245518607229)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:59.175622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:59.257809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16955da09561483387b1245518607229'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.269751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16955da09561483387b1245518607229'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.271061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16955da09561483387b1245518607229'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.304301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16955da09561483387b1245518607229'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.321769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16955da09561483387b1245518607229'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.356006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1028/2059] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:59 2022] DELETE /identity/v3/users/16955da09561483387b1245518607229 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:59.361936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:59.362224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:59.362449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:59.405574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:59.417547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:05:59.418775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:59.419136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:59.454770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:59.544435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aad76-9cdc-4b54-a1fc-333921e30da7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:59.551031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:59.551680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:59.552117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:59.552372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] PATH_INFO: `/v3/projects/d71daf73ae764e8a9ccd07a31cbad6f7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:59.554743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71daf73ae764e8a9ccd07a31cbad6f7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:59.557894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:59.579559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:59.580810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1032/2060] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:05:59.592608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:59.594224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:59.632314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71daf73ae764e8a9ccd07a31cbad6f7', 'request_id': 'req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.640074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71daf73ae764e8a9ccd07a31cbad6f7', 'request_id': 'req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.647909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71daf73ae764e8a9ccd07a31cbad6f7', 'request_id': 'req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.674057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] The token cache is being invalidate because project d71daf73ae764e8a9ccd07a31cbad6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:05:59.674351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.677022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.679726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1029/2061] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:05:59 2022] DELETE /identity/v3/projects/d71daf73ae764e8a9ccd07a31cbad6f7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:05:59.685388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:05:59.687235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5230d4-0f6e-4791-96fc-fef70acb58eb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:59.690640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:59.821012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:59.821885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:59.822164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:59.822435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:59.826518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:05:59.833261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefab7a2-63a9-4d03-a0c7-91c2ed9bf4c2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:05:59.839108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:05:59.839755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:05:59.840144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:05:59.840309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] PATH_INFO: `/v3/users/d5adfbc9159949dd93c8e1e406cc1bc7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:05:59.841839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5adfbc9159949dd93c8e1e406cc1bc7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:59.844015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:59.851462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:05:59.859495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:05:59.884762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1033/2062] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:05:59.965316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5adfbc9159949dd93c8e1e406cc1bc7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.984540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5adfbc9159949dd93c8e1e406cc1bc7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:05:59.985313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5adfbc9159949dd93c8e1e406cc1bc7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.002451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5adfbc9159949dd93c8e1e406cc1bc7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.032847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5adfbc9159949dd93c8e1e406cc1bc7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.056732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1030/2063] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:05:59 2022] DELETE /identity/v3/users/d5adfbc9159949dd93c8e1e406cc1bc7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:00.372032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:00.373776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.387666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da582fa-e9c1-4e93-af5c-97dcbd907451 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:00.426206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:00.427631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.466511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:00.497457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:00.498109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.498382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.498656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] PATH_INFO: `/v3/users/9cfd048f992343a88e245c6e30fcfe3d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.501697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cfd048f992343a88e245c6e30fcfe3d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:00.503768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:00.518341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fa96a-fbf2-4eb0-8c1d-4730cc5246b5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:00.530153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:00.531108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.531478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.531875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] PATH_INFO: `/v3/projects/d542e5d2c86d44758a8577e202abe81c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.533143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d542e5d2c86d44758a8577e202abe81c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:00.535448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:00.541873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:00.583530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cfd048f992343a88e245c6e30fcfe3d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.607193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cfd048f992343a88e245c6e30fcfe3d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.608045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cfd048f992343a88e245c6e30fcfe3d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.623651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cfd048f992343a88e245c6e30fcfe3d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.629676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd542e5d2c86d44758a8577e202abe81c', 'request_id': 'req-a8830d91-ca95-473c-bb8b-a8ec8caba253'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.639847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd542e5d2c86d44758a8577e202abe81c', 'request_id': 'req-a8830d91-ca95-473c-bb8b-a8ec8caba253'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.643381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cfd048f992343a88e245c6e30fcfe3d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.648447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd542e5d2c86d44758a8577e202abe81c', 'request_id': 'req-a8830d91-ca95-473c-bb8b-a8ec8caba253'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.652498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1034/2064] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:00 2022] DELETE /identity/v3/users/9cfd048f992343a88e245c6e30fcfe3d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:00.656207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:00.657985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.666599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] The token cache is being invalidate because project d542e5d2c86d44758a8577e202abe81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:00.666904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.667742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a8830d91-ca95-473c-bb8b-a8ec8caba253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.669710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1031/2065] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:00 2022] DELETE /identity/v3/projects/d542e5d2c86d44758a8577e202abe81c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:00.671844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.672349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.673165 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.702730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:00.704001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.767763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a8013-34ef-4a1e-8419-15c84bca0c94 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:00.773108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:00.773844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.774222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.774593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] PATH_INFO: `/v3/projects/98d68a64542b4b33872afe9a8c8ce8ef` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.776046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d68a64542b4b33872afe9a8c8ce8ef)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:00.778747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:00.783905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1032/2066] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:00.797736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:00.799887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.837413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d68a64542b4b33872afe9a8c8ce8ef', 'request_id': 'req-9bcbf401-0016-4c20-a8cd-bb2594310d8e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.845627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d68a64542b4b33872afe9a8c8ce8ef', 'request_id': 'req-9bcbf401-0016-4c20-a8cd-bb2594310d8e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.855387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d68a64542b4b33872afe9a8c8ce8ef', 'request_id': 'req-9bcbf401-0016-4c20-a8cd-bb2594310d8e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.866285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148e48c-e522-4f65-86cb-80845c8c5bb1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:00.869540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] The token cache is being invalidate because project 98d68a64542b4b33872afe9a8c8ce8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:00.869838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.870518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9bcbf401-0016-4c20-a8cd-bb2594310d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:00.872508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1035/2067] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:00 2022] DELETE /identity/v3/projects/98d68a64542b4b33872afe9a8c8ce8ef => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:00.877059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:00.877450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a2a2-b34d-4546-a879-388cb8c3e697 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.877938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a2a2-b34d-4546-a879-388cb8c3e697 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.878057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31a2a2-b34d-4546-a879-388cb8c3e697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.898902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7f31a2a2-b34d-4546-a879-388cb8c3e697 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:00.900126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7f31a2a2-b34d-4546-a879-388cb8c3e697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.922516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:00.923115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.923395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.923672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.925455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.927488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:00.963507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1036/2068] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:00.971310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:00.971598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:00.971872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:00.991498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:00.992692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:00.993667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:00.995160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.000911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1037/2069] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:01.005193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.005590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1033/2070] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:01.014631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a0f4-0201-49e3-83a6-e4ec49029a30 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.020791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.022930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.022930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.024353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.025246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:01.025362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.026928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.043474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1038/2071] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:01.047176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.057759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26edcf-85e5-4719-a1d6-892dd396340b service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.062593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.063191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.063854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.063854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.064818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.066741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.101987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1034/2072] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:01 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:01.106415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.122272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933b29b-47e3-46db-ade2-d50ef2f6cf83 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.201731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.202608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.202895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.203225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.204010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.206140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.260682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1039/2073] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:01 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:01.264322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.277310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb30eb4-2c1d-48d8-bb22-265bc48ef00b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.288151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.289728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.290201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.291414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.292978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.295039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.334703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.334884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1035/2074] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:01.353797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0d6d-abc2-4272-a39e-b0f3bc2683df admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.361569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.362504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.363029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.363562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.364544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.368831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.392587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.394643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:01.399159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1040/2075] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:01.409797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.421874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021df0a-a53c-47d5-b57b-7b4a298a9cf0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.426721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.427483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.427799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.428108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] PATH_INFO: `/v3/projects/dccd0b5f434a45e78fd13eb76cac9a27` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.429471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccd0b5f434a45e78fd13eb76cac9a27)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.431393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.432820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2037e0-2efa-4e60-ac23-a5a0fab51bc2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.441959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03b617-6f12-4d5a-aca9-89c2e8a71f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.442865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b617-6f12-4d5a-aca9-89c2e8a71f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.443157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b617-6f12-4d5a-aca9-89c2e8a71f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.443955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03b617-6f12-4d5a-aca9-89c2e8a71f12 admin admin] PATH_INFO: `/v3/projects/49e9f9f55c324d0bb63b08d8a93c8450/users/f5078251c73143bc9801ef0d50782dbc/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.462832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03b617-6f12-4d5a-aca9-89c2e8a71f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e9f9f55c324d0bb63b08d8a93c8450, user_id=f5078251c73143bc9801ef0d50782dbc, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.465662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03b617-6f12-4d5a-aca9-89c2e8a71f12 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.483041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1041/2076] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:01 2022] PUT /identity/v3/projects/49e9f9f55c324d0bb63b08d8a93c8450/users/f5078251c73143bc9801ef0d50782dbc/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:01.494653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.494994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.495325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.498485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccd0b5f434a45e78fd13eb76cac9a27', 'request_id': 'req-b4bb0264-9242-43d5-a3da-1ef63981c755'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.507075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccd0b5f434a45e78fd13eb76cac9a27', 'request_id': 'req-b4bb0264-9242-43d5-a3da-1ef63981c755'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.515123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccd0b5f434a45e78fd13eb76cac9a27', 'request_id': 'req-b4bb0264-9242-43d5-a3da-1ef63981c755'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.532544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] MFA Rules not processed for user `f5078251c73143bc9801ef0d50782dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:01.533076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] The token cache is being invalidate because project dccd0b5f434a45e78fd13eb76cac9a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:01.533422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.533908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:01.534086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.535976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1036/2077] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:01 2022] DELETE /identity/v3/projects/dccd0b5f434a45e78fd13eb76cac9a27 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:01.539768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.541698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:01.555218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:01.614215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:01.620235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb0264-9242-43d5-a3da-1ef63981c755 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.626495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1042/2078] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:01.630680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.630815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.631839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.632121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.632372 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] PATH_INFO: `/v3/users/648927cc23614a77897534adaafaa9d7` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.632960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:01.634238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=648927cc23614a77897534adaafaa9d7)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.636373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.649323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8a345-6be2-466e-9321-bc7488318286 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.654099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.654705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.654984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.655261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.655894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.657841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.687118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1043/2079] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:01 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:01.689267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648927cc23614a77897534adaafaa9d7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.691241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.700780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648927cc23614a77897534adaafaa9d7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.701584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648927cc23614a77897534adaafaa9d7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.716465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648927cc23614a77897534adaafaa9d7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.724125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648927cc23614a77897534adaafaa9d7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:01.730246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1037/2080] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:01 2022] DELETE /identity/v3/users/648927cc23614a77897534adaafaa9d7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:01.778738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58746909-5a25-425f-91b4-779ffd8b11ee admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.842822 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.843429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.843705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.843974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.844656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.846592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.868131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1044/2081] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:01 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:01.872064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.873516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:01.890101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f1fda-117b-451a-9d13-486fe46149bb admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.893281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.893835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.894109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.894380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.895015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.896951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.910277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1038/2082] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:01.911441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.921694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c45de-9dde-4ba0-ab17-2906fc26a4d6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.926390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340679a2-fa45-483e-a6c7-f742967ed0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.926990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340679a2-fa45-483e-a6c7-f742967ed0cd admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.927272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340679a2-fa45-483e-a6c7-f742967ed0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.927548 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340679a2-fa45-483e-a6c7-f742967ed0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.928859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340679a2-fa45-483e-a6c7-f742967ed0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.930466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340679a2-fa45-483e-a6c7-f742967ed0cd admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:01.941672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1045/2083] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:01.945841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:01.956280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566153b7-e5b6-4c57-91d0-d2ae5a1bce01 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:01.962474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:01.963168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:01.963448 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:01.963725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] PATH_INFO: `/v3/projects/7533c690d78c41b2ad8b882e1a16b2b1/users/7e62c5edfa524e05a3f1e94b79f76c57/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:01.988628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7533c690d78c41b2ad8b882e1a16b2b1, user_id=7e62c5edfa524e05a3f1e94b79f76c57, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:01.991598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.005227 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1039/2084] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:01 2022] PUT /identity/v3/projects/7533c690d78c41b2ad8b882e1a16b2b1/users/7e62c5edfa524e05a3f1e94b79f76c57/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:02.016352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.016638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.016939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.034737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.036727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.054776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] MFA Rules not processed for user `7e62c5edfa524e05a3f1e94b79f76c57`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:02.056221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.094069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.109222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.110619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dcbb8-92e7-4db8-927c-46223ad865d0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.116089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.116742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.117026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.117434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] PATH_INFO: `/v3/projects/38e57fbc11c74203b75b8e98be2e5925` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.119385 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e57fbc11c74203b75b8e98be2e5925)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.121394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.130698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1046/2085] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:02.135664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.146418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef64a0d-44a1-4a3c-954b-35f5dee65acc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.151656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.152536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.152855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.153321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.154052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.156104 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.173909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e57fbc11c74203b75b8e98be2e5925', 'request_id': 'req-b6324d59-6034-4a41-a84d-ea0486edfb35'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.181318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e57fbc11c74203b75b8e98be2e5925', 'request_id': 'req-b6324d59-6034-4a41-a84d-ea0486edfb35'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.185747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1047/2086] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:02.188180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e57fbc11c74203b75b8e98be2e5925', 'request_id': 'req-b6324d59-6034-4a41-a84d-ea0486edfb35'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.189775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.202148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] The token cache is being invalidate because project 38e57fbc11c74203b75b8e98be2e5925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:02.202465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.203142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b6324d59-6034-4a41-a84d-ea0486edfb35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.204772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1040/2087] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:02 2022] DELETE /identity/v3/projects/38e57fbc11c74203b75b8e98be2e5925 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:02.209642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e503ff1-3f40-48b3-a139-b525f6d70c07 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.210064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e503ff1-3f40-48b3-a139-b525f6d70c07 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.210360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e503ff1-3f40-48b3-a139-b525f6d70c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.231328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2e503ff1-3f40-48b3-a139-b525f6d70c07 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:02.232535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2e503ff1-3f40-48b3-a139-b525f6d70c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.253477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf7766-ca47-4f11-bf51-6ca3359b3217 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.258699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.259294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.259573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.259840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.260457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.262416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.284553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1048/2088] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:02.289243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.290802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.293577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1041/2089] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:02.303584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237778-5365-4574-b716-7bb549bc2a17 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.306282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.306575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.306862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.308707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.309319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.309598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.309880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.310524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.312122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.322932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1049/2090] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:02.326333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:02.326993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.327545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.336132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1042/2091] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:02.337654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a765b75-3fca-4017-a135-413cfa7fa697 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.341631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.342854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.343571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.343853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.344587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] PATH_INFO: `/v3/projects/f4ac80f2c9b146cea4e9805a52e6513c/users/7070192188964234be64c0d1d0e1d089/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.353607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559348d-a72f-41b1-afb0-dbd6d7bea775 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.359686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.360310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.360576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.360942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.361817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:02.362121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.362390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ac80f2c9b146cea4e9805a52e6513c, user_id=7070192188964234be64c0d1d0e1d089, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.363742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.365100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.384958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1043/2092] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:02.391249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1050/2093] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:02 2022] PUT /identity/v3/projects/f4ac80f2c9b146cea4e9805a52e6513c/users/7070192188964234be64c0d1d0e1d089/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:02.391427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.392684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.401138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec8647-431f-4cf5-94cd-76d6105b5a07 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.472142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d34dd-752d-4316-b4a6-1c836b220bc2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.477855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.478789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.479119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.479477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.480225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.482016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.487427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.488272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.488395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.488598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.489481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.492388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.497488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1051/2094] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:02.498433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.500242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.502895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.518044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2272-beb3-4a31-8df2-ccb55c6e7848 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.525258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.525896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.526190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.526507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] PATH_INFO: `/v3/users/c26cd1261d904eaeba85ff5209518e9d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.527664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26cd1261d904eaeba85ff5209518e9d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.529773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.531000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1044/2095] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:02.532042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.547658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.589323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26cd1261d904eaeba85ff5209518e9d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.602501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26cd1261d904eaeba85ff5209518e9d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.604015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b13d4-ddb2-4094-82d0-1ceb85226c6a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.604660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26cd1261d904eaeba85ff5209518e9d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.610304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.611615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.612132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.612304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] PATH_INFO: `/v3/projects/f4ac80f2c9b146cea4e9805a52e6513c/users/7070192188964234be64c0d1d0e1d089/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.613115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26cd1261d904eaeba85ff5209518e9d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.614525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ac80f2c9b146cea4e9805a52e6513c, user_id=7070192188964234be64c0d1d0e1d089, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.618342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.629791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1045/2096] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:02 2022] PUT /identity/v3/projects/f4ac80f2c9b146cea4e9805a52e6513c/users/7070192188964234be64c0d1d0e1d089/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:02.631054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-85d07d11-b874-42f8-a545-6434a41dc20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26cd1261d904eaeba85ff5209518e9d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:02.633878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.635800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.642000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1052/2097] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:02 2022] DELETE /identity/v3/users/c26cd1261d904eaeba85ff5209518e9d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:02.642000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd93df-af16-454a-b6d8-9d739d8e754d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.642000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd93df-af16-454a-b6d8-9d739d8e754d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.642000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd93df-af16-454a-b6d8-9d739d8e754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.687803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c2fd93df-af16-454a-b6d8-9d739d8e754d None None] MFA Rules not processed for user `7070192188964234be64c0d1d0e1d089`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:02.690411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c2fd93df-af16-454a-b6d8-9d739d8e754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.710530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc0493-dceb-474a-bc1d-625509cf7ff2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.783619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.784232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.784510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.784815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.785449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.787507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.791330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1053/2098] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:02.792742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.793121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.793490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.812862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1046/2099] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:02.813746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.814606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:02.815529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.815793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.826177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1054/2100] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:02.827015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.838181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c658eb-cc50-40f9-9e46-8ac32cc288f1 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.845159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.845878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.846164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.846437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.847115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.848505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.848982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.864211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1055/2101] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:02.865191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.867412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.891231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac27b84-0852-485e-8444-a73ae1122d07 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.895477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.896501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.896891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.897159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.899257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:02.928360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677fa5-67ba-4931-b422-f54c068cb96b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.933150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:02.933766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:02.934169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:02.934454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:02.944306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.945813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.951455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.965496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1056/2102] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:02.966160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:02.968146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:02.970012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:02.984401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d63e0-8794-4d38-a633-8083be8fb9a4 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:02.990680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d7fe2-2dfb-40dd-ab26-6fcbb0c31c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.004996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d7fe2-2dfb-40dd-ab26-6fcbb0c31c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.004996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d7fe2-2dfb-40dd-ab26-6fcbb0c31c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.004996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d7fe2-2dfb-40dd-ab26-6fcbb0c31c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.004996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d7fe2-2dfb-40dd-ab26-6fcbb0c31c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.005325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1047/2103] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:03.005419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d7fe2-2dfb-40dd-ab26-6fcbb0c31c47 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.005419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1057/2104] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:03.009656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.020922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625b3e3-db54-45e8-ba96-067623d5320e service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.026913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.027732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.028021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.028297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] PATH_INFO: `/v3/projects/8b3cb4bc9eac406b97cb9179b7017257/users/e2a7cb20dfc548f79b82140bff86457e/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.048260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3cb4bc9eac406b97cb9179b7017257, user_id=e2a7cb20dfc548f79b82140bff86457e, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.053045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.069323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1048/2105] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:03 2022] PUT /identity/v3/projects/8b3cb4bc9eac406b97cb9179b7017257/users/e2a7cb20dfc548f79b82140bff86457e/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:03.085050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.085050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.085465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.123018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] MFA Rules not processed for user `e2a7cb20dfc548f79b82140bff86457e`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:03.124244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:03.245528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1058/2106] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:03.247663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.335563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a25c1a-3371-4ae1-bf44-004542163da7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.361489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.362146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.362494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.362845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.363652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.366319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.427417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1049/2107] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:03 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:03.432600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.445345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06047916-3d0b-442f-a768-7fb196c25a6e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.472500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.475192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:03.539427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:03.559228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.560030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.560462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.561956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.561956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.564353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.603919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61663-b4d0-4824-bb05-b0b3edf4327c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.605717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1059/2108] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:03 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:03.611472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.632204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.633039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.633463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.633754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] PATH_INFO: `/v3/projects/0fa6567f562842c189f7d3c96d232675` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.644108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa6567f562842c189f7d3c96d232675)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.644315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75635dd8-ee0c-4bf8-8b0d-eb49e5a947b2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.645127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.656435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.657241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.657653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.658028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.658788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.660610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.685719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1060/2109] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:03.690114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.721524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70d984-9feb-4688-b868-09315c1eed66 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.732845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa6567f562842c189f7d3c96d232675', 'request_id': 'req-a4fe4a19-4b9c-4bce-8200-4c421aab8277'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.767267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa6567f562842c189f7d3c96d232675', 'request_id': 'req-a4fe4a19-4b9c-4bce-8200-4c421aab8277'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.792758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa6567f562842c189f7d3c96d232675', 'request_id': 'req-a4fe4a19-4b9c-4bce-8200-4c421aab8277'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.811710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:03.823880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.825421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.826265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.827399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] PATH_INFO: `/v3/projects/dc70db8e94af4e83a201c0cecfdb5efb/users/db30afa57e524e12925f9b496c4da682/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.831043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] The token cache is being invalidate because project 0fa6567f562842c189f7d3c96d232675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:03.831331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.833046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.837163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1050/2110] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:03 2022] DELETE /identity/v3/projects/0fa6567f562842c189f7d3c96d232675 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:03.841093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.842960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:03.864604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc70db8e94af4e83a201c0cecfdb5efb, user_id=db30afa57e524e12925f9b496c4da682, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.866790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe4a19-4b9c-4bce-8200-4c421aab8277 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.873290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.882272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:03.883035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:03.883314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:03.883589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] PATH_INFO: `/v3/users/9d897c33e85a4a8fb791d1a1050cf007` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:03.885567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d897c33e85a4a8fb791d1a1050cf007)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:03.891762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:03.905558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1061/2111] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:03 2022] PUT /identity/v3/projects/dc70db8e94af4e83a201c0cecfdb5efb/users/db30afa57e524e12925f9b496c4da682/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:03.907513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:03.909470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:03.979143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d897c33e85a4a8fb791d1a1050cf007'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.998301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90af6b-8a25-4982-bb29-9dd5adaaed39 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:03.998587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d897c33e85a4a8fb791d1a1050cf007'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:03.999762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d897c33e85a4a8fb791d1a1050cf007'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.005278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:04.006918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.007301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.007820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.009472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:04.011314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:04.027962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d897c33e85a4a8fb791d1a1050cf007'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.036314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1062/2112] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:04.041081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3a3352d1-d8a4-49f8-a278-76f6ef7f2567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d897c33e85a4a8fb791d1a1050cf007'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.041786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:04.043683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.059396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1051/2113] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:03 2022] DELETE /identity/v3/users/9d897c33e85a4a8fb791d1a1050cf007 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:04.086623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c915910-d994-4518-b9f1-1e15954a5cd8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:04.213126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:04.214224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.214710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.215129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] PATH_INFO: `/v3/projects/dc70db8e94af4e83a201c0cecfdb5efb/users/db30afa57e524e12925f9b496c4da682/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.218127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc70db8e94af4e83a201c0cecfdb5efb, user_id=db30afa57e524e12925f9b496c4da682, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:04.222049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:04.242937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1063/2114] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:04 2022] PUT /identity/v3/projects/dc70db8e94af4e83a201c0cecfdb5efb/users/db30afa57e524e12925f9b496c4da682/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:04.258219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.258580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.258860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.296033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] MFA Rules not processed for user `db30afa57e524e12925f9b496c4da682`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:04.297999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.336118 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:04.404161 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1052/2115] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:04.411810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:04.414412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.510891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e93e5-a017-43d5-803f-6f9c5e473021 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:04.518943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:04.519699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.520021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.520475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.531846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:04.533412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:04.539241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:04.541006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.570295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1064/2116] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:04.636835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c692-0d3c-428d-8335-a0275ff9f8b8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:04.654072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:04.654980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.655721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.656726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] PATH_INFO: `/v3/projects/cf0f97ecb8294f819c797bde0325b1bc` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.658511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0f97ecb8294f819c797bde0325b1bc)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:04.660603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:04.735269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:04.736814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.749482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0f97ecb8294f819c797bde0325b1bc', 'request_id': 'req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.750472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab888b-1fa8-4fc0-87c9-c067a9f0d4f4 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:04.755488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0f97ecb8294f819c797bde0325b1bc', 'request_id': 'req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.764918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0f97ecb8294f819c797bde0325b1bc', 'request_id': 'req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.778818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:04.785869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] The token cache is being invalidate because project cf0f97ecb8294f819c797bde0325b1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:04.786171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.787835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4e5c29f5-9848-41ee-8d8e-0a7c566dd278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:04.789871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1053/2117] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:04 2022] DELETE /identity/v3/projects/cf0f97ecb8294f819c797bde0325b1bc => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:04.794772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6351-ccb4-4a05-a84c-68b55ab31710 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.795321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6351-ccb4-4a05-a84c-68b55ab31710 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.795751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b6351-ccb4-4a05-a84c-68b55ab31710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.822456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-082b6351-ccb4-4a05-a84c-68b55ab31710 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:04.823718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-082b6351-ccb4-4a05-a84c-68b55ab31710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.848705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:04.850012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.850411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.850785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.853233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.889096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:04.907620 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1054/2118] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:04.917131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.917912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.918489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.932207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:04.934143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:04.940937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:04.942149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:04.944705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1065/2119] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:04.956419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1055/2120] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:04.957622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:04.971609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65680738-faca-4682-954b-904822c15ec0 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:04.976309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:04.976982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:04.977264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:04.977542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:04.978765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:04.979091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:04.980937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:04.997138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1066/2121] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:04.999997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.012354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc556d-fa81-46a4-b7f2-8974b21859da None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.017500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.018115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.018394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.018670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.019354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.021631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.064808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1056/2122] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:04 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:05.067105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.079458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb66a6-3b08-4a8e-a557-bffd2a1f8aeb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.153079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.153713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.153954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.154232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.154966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.156839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.180725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1067/2123] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:05 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:05.184275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.195421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0e5aa-babb-4a9a-9ff6-372f69807330 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.204336 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.204979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.205261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.205536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.206342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.207951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.219178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1057/2124] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:05.230646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.244054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce8126-6feb-40d2-b36f-92b7662b6f25 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.249114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbf6c2-46d0-4baf-8f1e-12b4f29cfc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.249760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf6c2-46d0-4baf-8f1e-12b4f29cfc5b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.250000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf6c2-46d0-4baf-8f1e-12b4f29cfc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.250255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf6c2-46d0-4baf-8f1e-12b4f29cfc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.250901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbf6c2-46d0-4baf-8f1e-12b4f29cfc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.252758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbf6c2-46d0-4baf-8f1e-12b4f29cfc5b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.263914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1068/2125] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:05.267825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.278756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70efc4-0a44-41b0-9ee8-967244cdfa5d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.284286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.284995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.285402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.285681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] PATH_INFO: `/v3/projects/64c2866bc8cd49348487702c71865a1a/users/61b2ea9567664c60bda3d3d06b89dccd/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.309916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c2866bc8cd49348487702c71865a1a, user_id=61b2ea9567664c60bda3d3d06b89dccd, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.312481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.344271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.344564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.344900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.348699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1058/2126] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:05 2022] PUT /identity/v3/projects/64c2866bc8cd49348487702c71865a1a/users/61b2ea9567664c60bda3d3d06b89dccd/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:05.416220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] MFA Rules not processed for user `61b2ea9567664c60bda3d3d06b89dccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:05.417471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:05.455270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:05.562380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1069/2127] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:05.579405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.581924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:05.654057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.655502 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:05.675917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01950d-43b8-4e8e-9a32-f8ffc3f1a30c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.679702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.680370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.680743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.681093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.690283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.694619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.705398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1059/2128] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:05.706293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:05.707942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:05.743055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05564c-c091-413b-8aef-ae0243f0c0b8 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.750254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.751256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.751686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.752108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] PATH_INFO: `/v3/users/e2a7cb20dfc548f79b82140bff86457e` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.753535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a7cb20dfc548f79b82140bff86457e)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.757167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.799345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e8771-9e99-4d5c-bf1a-e0881b609239 service glance] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:05.808236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:05.809107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.809563 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.810019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.812769 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:05.837953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a7cb20dfc548f79b82140bff86457e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:05.854174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a7cb20dfc548f79b82140bff86457e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:05.855904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a7cb20dfc548f79b82140bff86457e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:05.867623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a7cb20dfc548f79b82140bff86457e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:05.884387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-07a0c7a4-566d-4494-be83-4a6340c15812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a7cb20dfc548f79b82140bff86457e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:05.887157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:05.897356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1070/2129] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:05 2022] DELETE /identity/v3/users/e2a7cb20dfc548f79b82140bff86457e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:05.901728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:05.902017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:05.902296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:05.909860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:05.912923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:05.922831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:05.924118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:05.924488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1060/2130] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:05.964465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:05.995553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1071/2131] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:06.006112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:06.008042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:06.075796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a853dcd3-19df-470e-aff1-d4cfebf9af39 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:06.157935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:06.158662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:06.159082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:06.159474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:06.172865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:06.175011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:06.190579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1061/2132] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:06.640276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:06.641979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:06.657466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebb08d-9d95-4c90-9ff2-12a5714f6ec8 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:06.664210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:06.665011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:06.665535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:06.665911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] PATH_INFO: `/v3/projects/8b3cb4bc9eac406b97cb9179b7017257` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:06.667219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3cb4bc9eac406b97cb9179b7017257)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:06.669268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:06.739391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3cb4bc9eac406b97cb9179b7017257', 'request_id': 'req-739c72a5-8581-48c4-a7ad-15e05e5144cb'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.747790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3cb4bc9eac406b97cb9179b7017257', 'request_id': 'req-739c72a5-8581-48c4-a7ad-15e05e5144cb'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.760983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3cb4bc9eac406b97cb9179b7017257', 'request_id': 'req-739c72a5-8581-48c4-a7ad-15e05e5144cb'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.772722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] The token cache is being invalidate because project 8b3cb4bc9eac406b97cb9179b7017257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:06.773012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.773696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.778192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1072/2133] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:06 2022] DELETE /identity/v3/projects/8b3cb4bc9eac406b97cb9179b7017257 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:06.779363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:06.789895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b255e7e-2369-41c4-a8b5-04b955fcd1e8 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:06.851184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:06.851813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:06.852092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:06.852364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] PATH_INFO: `/v3/users/db30afa57e524e12925f9b496c4da682` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:06.853523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db30afa57e524e12925f9b496c4da682)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:06.855454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:06.902225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:06.906122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db30afa57e524e12925f9b496c4da682'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.920196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db30afa57e524e12925f9b496c4da682'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.921173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db30afa57e524e12925f9b496c4da682'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.933705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db30afa57e524e12925f9b496c4da682'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.940641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d8f571bd-8dbb-45a8-81fd-5f8c98449aa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db30afa57e524e12925f9b496c4da682'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:06.947295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1062/2134] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:06 2022] DELETE /identity/v3/users/db30afa57e524e12925f9b496c4da682 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:08.039670 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.041212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:08.054757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c72a5-8581-48c4-a7ad-15e05e5144cb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.112799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.113429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.113706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.113978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] PATH_INFO: `/v3/projects/dc70db8e94af4e83a201c0cecfdb5efb` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.115681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc70db8e94af4e83a201c0cecfdb5efb)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.117613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.187520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc70db8e94af4e83a201c0cecfdb5efb', 'request_id': 'req-531e19d4-9cf2-45cb-866e-9ea250f309e0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:08.194309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc70db8e94af4e83a201c0cecfdb5efb', 'request_id': 'req-531e19d4-9cf2-45cb-866e-9ea250f309e0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:08.202928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc70db8e94af4e83a201c0cecfdb5efb', 'request_id': 'req-531e19d4-9cf2-45cb-866e-9ea250f309e0'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:08.208029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:08.217578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] The token cache is being invalidate because project dc70db8e94af4e83a201c0cecfdb5efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:08.218024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:08.219014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-531e19d4-9cf2-45cb-866e-9ea250f309e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:08.221593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1073/2135] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:08 2022] DELETE /identity/v3/projects/dc70db8e94af4e83a201c0cecfdb5efb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:08.227358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.227899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.228023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.248300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:08.249532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:08.310449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1063/2136] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:08.318200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.318561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.318937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.339124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:08.340358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:08.348619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1074/2137] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:08.353831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.363554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb8799-5f9a-4d57-b4dc-6cc84953f18b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.375891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.375891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.375891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.375891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.376261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:08.376261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.376261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.396069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1064/2138] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:08.399290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.410144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e3ee3-a36d-4fc2-9646-5f892ee5d0f2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.416956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.417624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.417898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.418170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.418813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.420789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.460087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1075/2139] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:08.464484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.476935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336f49-10a9-4353-b52c-8fcbe10f075c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.551902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.553935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.553935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.553935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.554577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.557314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.592543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.602318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1065/2140] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:08.608483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b78ce8-3915-4f4d-86e8-c800d3609e1f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.611596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.612265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.612655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.613022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.613874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.616291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.627538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1076/2141] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:08.632429 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.642451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1e1d9-2c6f-4060-84a6-6e22bb82ea8e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.647462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575db175-19ee-4015-a5bd-c3b1cf93f25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.648069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575db175-19ee-4015-a5bd-c3b1cf93f25c admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.648356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575db175-19ee-4015-a5bd-c3b1cf93f25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.648634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575db175-19ee-4015-a5bd-c3b1cf93f25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.649307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575db175-19ee-4015-a5bd-c3b1cf93f25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.650909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575db175-19ee-4015-a5bd-c3b1cf93f25c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.664748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1066/2142] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:08.669043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:08.680920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ced37-71e6-4c92-8184-ae85f34e5511 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:08.690012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:08.691325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.691752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.692203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] PATH_INFO: `/v3/projects/40a509722dda42c5aa4e4c07f5aa28aa/users/5ccc1a936e6d4c51af39b8edc7acf63d/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.713506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a509722dda42c5aa4e4c07f5aa28aa, user_id=5ccc1a936e6d4c51af39b8edc7acf63d, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:08.717585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:08.745305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1077/2143] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:08 2022] PUT /identity/v3/projects/40a509722dda42c5aa4e4c07f5aa28aa/users/5ccc1a936e6d4c51af39b8edc7acf63d/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:08.752186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:08.752588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:08.753234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:08.796146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] MFA Rules not processed for user `5ccc1a936e6d4c51af39b8edc7acf63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:08.797493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:08.820017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:08.904359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1067/2144] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:08.910001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.007719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f633fd0-fb21-497b-bb26-3eb0842d7098 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.014301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.014946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.015231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.015512 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.016192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.018164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.048698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1078/2145] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:08 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:09.053144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.064123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e40eb-c7ab-4104-a7a9-2306811ba674 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.126556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.127464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.127907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.128343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.130172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.131018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.159369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1068/2146] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:09 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:09.164260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.176730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251a8f2-3cd2-40f6-a58d-eff6f4f7ada5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.184395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.184395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.184395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.184395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.185365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.186935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.201851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1079/2147] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:09.203619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.237120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868fe7c-c80d-4dd3-a4e4-a7431b80bd5d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.251117 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.251794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.252072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.252348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] PATH_INFO: `/v3/projects/457f398387624dcdab4c8aa95b08f222/users/4f8be8bf511c47bdaccf5832695ac26b/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.277818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457f398387624dcdab4c8aa95b08f222, user_id=4f8be8bf511c47bdaccf5832695ac26b, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.280965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.297949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1069/2148] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:09 2022] PUT /identity/v3/projects/457f398387624dcdab4c8aa95b08f222/users/4f8be8bf511c47bdaccf5832695ac26b/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:09.301962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.313175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0704e-9acb-4ce4-9692-c2f60690ca5a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.388209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734aab0-bbcb-4932-9b4f-fc7358128dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.388775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734aab0-bbcb-4932-9b4f-fc7358128dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.389060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734aab0-bbcb-4932-9b4f-fc7358128dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.389337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734aab0-bbcb-4932-9b4f-fc7358128dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.389990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734aab0-bbcb-4932-9b4f-fc7358128dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.391588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734aab0-bbcb-4932-9b4f-fc7358128dd4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.403439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1080/2149] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:09.411566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.426363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbb5aa-9b18-43dd-8aa4-22122c1ce8a5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.432609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.433357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.433763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.434020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] PATH_INFO: `/v3/projects/457f398387624dcdab4c8aa95b08f222/users/4f8be8bf511c47bdaccf5832695ac26b/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.436138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457f398387624dcdab4c8aa95b08f222, user_id=4f8be8bf511c47bdaccf5832695ac26b, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.438872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.452626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1070/2150] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:09 2022] PUT /identity/v3/projects/457f398387624dcdab4c8aa95b08f222/users/4f8be8bf511c47bdaccf5832695ac26b/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:09.464297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.464631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.464900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.477381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:09.515493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] MFA Rules not processed for user `4f8be8bf511c47bdaccf5832695ac26b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:09.517315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:09.611923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1081/2151] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:09.636045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:09.638026 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:09.731289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9325be-65c3-4939-9c68-8b74fe787422 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:09.748042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:09.748694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:09.748995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:09.749726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:09.766822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:09.768374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:09.779346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1071/2152] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:10.063620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.065418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.163598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc60d7a-553b-40c3-9187-67ec64cbde00 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.173603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.175587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.175875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.176239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] PATH_INFO: `/v3/users/f5078251c73143bc9801ef0d50782dbc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.177629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5078251c73143bc9801ef0d50782dbc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.180234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.262105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5078251c73143bc9801ef0d50782dbc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.274297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5078251c73143bc9801ef0d50782dbc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.278960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.279845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5078251c73143bc9801ef0d50782dbc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.289214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38febf-077c-457b-824b-1ba67eec4330 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.292751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5078251c73143bc9801ef0d50782dbc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.294781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f8be8bf511c47bdaccf5832695ac26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457f398387624dcdab4c8aa95b08f222', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.295575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.296005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.296442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.297268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:10.297884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.300648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.309486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1faeff94-96d1-4792-b70f-163b08528765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5078251c73143bc9801ef0d50782dbc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.310413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:10.318811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1082/2153] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:10 2022] DELETE /identity/v3/users/f5078251c73143bc9801ef0d50782dbc => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:10.322612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1072/2154] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:06:10.323701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.324815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.325289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.325768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.326266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.343468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:10.345217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c3ade-ca13-43b1-94b1-d882f7031304 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.348993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:10.352655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.429474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f8be8bf511c47bdaccf5832695ac26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457f398387624dcdab4c8aa95b08f222', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.430499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.430797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.431074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.432142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.434265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.442198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1083/2155] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:10.452637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.453755 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:10.485928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1073/2156] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:10 2022] POST /identity/v3/projects => generated 354 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:10.490667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.540091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e53c12-c8c1-491a-a8c4-8778148ae193 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.582651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04489cf-52a0-496e-95a9-7b9672a42578 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.591979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f8be8bf511c47bdaccf5832695ac26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457f398387624dcdab4c8aa95b08f222', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.592822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.593260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.593636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.594488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:10.595248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.598355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.619101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1074/2157] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:06:10.625877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.627215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.627599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.628539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.662271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:10.674598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bfb74-34b4-4e87-a56a-c003b2cdd2f1 tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:10.680355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f8be8bf511c47bdaccf5832695ac26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457f398387624dcdab4c8aa95b08f222', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:10.681016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:10.681300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:10.681577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] PATH_INFO: `/v3/projects/7f5d0e24d39242e984473642db9c8e14` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:10.682707 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7f5d0e24d39242e984473642db9c8e14)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.684622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.704275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:10.705821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:10.718325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1084/2158] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:10.749211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5d0e24d39242e984473642db9c8e14', 'request_id': 'req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.756932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5d0e24d39242e984473642db9c8e14', 'request_id': 'req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.765479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5d0e24d39242e984473642db9c8e14', 'request_id': 'req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.791128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:10.796626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] The token cache is being invalidate because project 7f5d0e24d39242e984473642db9c8e14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:10.797012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.797830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:10.799901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1075/2159] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:10 2022] DELETE /identity/v3/projects/7f5d0e24d39242e984473642db9c8e14 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:11.061510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.063306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.080724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.081990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.088465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f065a06-28c5-4c77-8ef1-4dabd770b9c5 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.113343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:11.165145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea295fa-d85b-4946-8e8e-777f4c57b4ff tempest-VolumeQuotaClassesTest-1991143415 tempest-VolumeQuotaClassesTest-1991143415-project-admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.170925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.171670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.172041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.172398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] PATH_INFO: `/v3/projects/49e9f9f55c324d0bb63b08d8a93c8450` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.173632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e9f9f55c324d0bb63b08d8a93c8450)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.175416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.175699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.178589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.178950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.179292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] PATH_INFO: `/v3/users/5ccc1a936e6d4c51af39b8edc7acf63d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.180730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccc1a936e6d4c51af39b8edc7acf63d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.186380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.234809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e9f9f55c324d0bb63b08d8a93c8450', 'request_id': 'req-a672afe4-c47d-426b-a4a9-579e0d3045b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.241238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e9f9f55c324d0bb63b08d8a93c8450', 'request_id': 'req-a672afe4-c47d-426b-a4a9-579e0d3045b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.252204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e9f9f55c324d0bb63b08d8a93c8450', 'request_id': 'req-a672afe4-c47d-426b-a4a9-579e0d3045b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.256042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccc1a936e6d4c51af39b8edc7acf63d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.264757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccc1a936e6d4c51af39b8edc7acf63d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.265342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccc1a936e6d4c51af39b8edc7acf63d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.273449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] The token cache is being invalidate because project 49e9f9f55c324d0bb63b08d8a93c8450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:11.274226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.274470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.276962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccc1a936e6d4c51af39b8edc7acf63d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.279938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1076/2160] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:11 2022] DELETE /identity/v3/projects/49e9f9f55c324d0bb63b08d8a93c8450 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:11.280851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.282588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.291876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d2296985-3a0b-4fc2-b2dd-56de4959be3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccc1a936e6d4c51af39b8edc7acf63d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.298218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1085/2161] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:11 2022] DELETE /identity/v3/users/5ccc1a936e6d4c51af39b8edc7acf63d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:11.303044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.303676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.303962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.325319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:11.326243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.369490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672afe4-c47d-426b-a4a9-579e0d3045b1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.422611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1086/2162] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:11.429317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:11.438771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.440951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.460713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.461824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.462108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.462379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] PATH_INFO: `/v3/users/7e62c5edfa524e05a3f1e94b79f76c57` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.463905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e62c5edfa524e05a3f1e94b79f76c57)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.466190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.479002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:11.506256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62971f4-462b-418d-b9c4-4c64fe4640ec None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.510227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.510839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.511117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.511387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.521861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.523341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e62c5edfa524e05a3f1e94b79f76c57'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.523469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:11.531570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e62c5edfa524e05a3f1e94b79f76c57'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.532483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e62c5edfa524e05a3f1e94b79f76c57'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.533380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1087/2163] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:11.548146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e62c5edfa524e05a3f1e94b79f76c57'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.556301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e62c5edfa524e05a3f1e94b79f76c57'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:11.563423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1077/2164] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:11 2022] DELETE /identity/v3/users/7e62c5edfa524e05a3f1e94b79f76c57 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:11.882518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.884507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.917449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f944ef5-7d80-4e60-ac44-3de6e9294b27 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:11.940018 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:11.942188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:11.992368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:11.993313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:11.993744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:11.994176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] PATH_INFO: `/v3/projects/40a509722dda42c5aa4e4c07f5aa28aa` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:11.995441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a509722dda42c5aa4e4c07f5aa28aa)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:11.998772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:12.033963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ff4cf-b229-49fb-9fe6-24ad84e3a552 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:12.041889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:12.042539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:12.043075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:12.043075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] PATH_INFO: `/v3/projects/7533c690d78c41b2ad8b882e1a16b2b1` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:12.044251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7533c690d78c41b2ad8b882e1a16b2b1)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:12.046425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:12.056524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a509722dda42c5aa4e4c07f5aa28aa', 'request_id': 'req-faacb15b-4dba-4423-9c66-42c04226e24d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.063820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a509722dda42c5aa4e4c07f5aa28aa', 'request_id': 'req-faacb15b-4dba-4423-9c66-42c04226e24d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.072194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a509722dda42c5aa4e4c07f5aa28aa', 'request_id': 'req-faacb15b-4dba-4423-9c66-42c04226e24d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.086097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] The token cache is being invalidate because project 40a509722dda42c5aa4e4c07f5aa28aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:12.087012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.087254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.088959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1088/2165] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:11 2022] DELETE /identity/v3/projects/40a509722dda42c5aa4e4c07f5aa28aa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:12.092990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:12.096244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:12.113476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7533c690d78c41b2ad8b882e1a16b2b1', 'request_id': 'req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.121958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7533c690d78c41b2ad8b882e1a16b2b1', 'request_id': 'req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.123417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:12.132797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7533c690d78c41b2ad8b882e1a16b2b1', 'request_id': 'req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.154633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] The token cache is being invalidate because project 7533c690d78c41b2ad8b882e1a16b2b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:12.155369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.156528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.159630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1078/2166] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:11 2022] DELETE /identity/v3/projects/7533c690d78c41b2ad8b882e1a16b2b1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:12.162756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:12.164143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:12.177106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacb15b-4dba-4423-9c66-42c04226e24d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:12.195110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:12.242877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae7ddc-b6bd-4892-bc6d-d94b3eea8340 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:12.248711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:12.250536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:12.250897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:12.251273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] PATH_INFO: `/v3/users/7070192188964234be64c0d1d0e1d089` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:12.253224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7070192188964234be64c0d1d0e1d089)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:12.255246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:12.261556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:12.262439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:12.262885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:12.263318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] PATH_INFO: `/v3/users/4f8be8bf511c47bdaccf5832695ac26b` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:12.264918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8be8bf511c47bdaccf5832695ac26b)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:12.267063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:12.319592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7070192188964234be64c0d1d0e1d089'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.329156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7070192188964234be64c0d1d0e1d089'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.330046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7070192188964234be64c0d1d0e1d089'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.345582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8be8bf511c47bdaccf5832695ac26b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.351621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7070192188964234be64c0d1d0e1d089'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.355738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8be8bf511c47bdaccf5832695ac26b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.356947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8be8bf511c47bdaccf5832695ac26b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.361539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7070192188964234be64c0d1d0e1d089'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.369812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8be8bf511c47bdaccf5832695ac26b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.373330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1079/2167] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:12 2022] DELETE /identity/v3/users/7070192188964234be64c0d1d0e1d089 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:12.391603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8be8bf511c47bdaccf5832695ac26b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:12.398997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1089/2168] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:12 2022] DELETE /identity/v3/users/4f8be8bf511c47bdaccf5832695ac26b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:12.818064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:12.822967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:12.888750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:12.890307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:12.932930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304e45-aa57-4684-99ef-53c91cddef08 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:12.943374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:13.000325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdf469-8ca7-48a7-85b7-f0db695d784c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.007156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.007849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.008131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.008409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] PATH_INFO: `/v3/projects/457f398387624dcdab4c8aa95b08f222` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.011279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=457f398387624dcdab4c8aa95b08f222)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.013266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.023268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.024197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.024565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.024967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] PATH_INFO: `/v3/projects/f4ac80f2c9b146cea4e9805a52e6513c` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.026204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ac80f2c9b146cea4e9805a52e6513c)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.028223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.056945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:13.075634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457f398387624dcdab4c8aa95b08f222', 'request_id': 'req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.082689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457f398387624dcdab4c8aa95b08f222', 'request_id': 'req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.091559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457f398387624dcdab4c8aa95b08f222', 'request_id': 'req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.092184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ac80f2c9b146cea4e9805a52e6513c', 'request_id': 'req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.098256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ac80f2c9b146cea4e9805a52e6513c', 'request_id': 'req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.103006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] The token cache is being invalidate because project 457f398387624dcdab4c8aa95b08f222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:13.103556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.104299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5e8b08e5-9a5e-4df6-86c0-44285218ffcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.104969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ac80f2c9b146cea4e9805a52e6513c', 'request_id': 'req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.106011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1090/2169] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:12 2022] DELETE /identity/v3/projects/457f398387624dcdab4c8aa95b08f222 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:13.110654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273b095-55da-47a1-9054-51f5e23e14f8 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.110965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273b095-55da-47a1-9054-51f5e23e14f8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.111216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273b095-55da-47a1-9054-51f5e23e14f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.118272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] The token cache is being invalidate because project f4ac80f2c9b146cea4e9805a52e6513c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:13.118437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.119010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-e9a4117c-f9bb-44fb-b0f8-e939059c08dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:13.120761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1080/2170] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:12 2022] DELETE /identity/v3/projects/f4ac80f2c9b146cea4e9805a52e6513c => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:13.128276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.128600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.128909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.139176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8273b095-55da-47a1-9054-51f5e23e14f8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:13.140397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8273b095-55da-47a1-9054-51f5e23e14f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:13.151791 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:13.153163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:13.202258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1091/2171] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:13.210660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2046-bb4e-484b-803f-466127e83ee7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.210948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2046-bb4e-484b-803f-466127e83ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.211235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2046-bb4e-484b-803f-466127e83ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.221924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1081/2172] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:13.242961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3a3f2046-bb4e-484b-803f-466127e83ee7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:13.244517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3a3f2046-bb4e-484b-803f-466127e83ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:13.256300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1092/2173] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:13.260901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.274928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac7b23-6361-403b-854e-114f321afaa2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.286311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.288034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.288328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.288608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.290301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:13.290610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.292997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.294261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.294617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.294959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.316885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1082/2174] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:13.320292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.322027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:13.323748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:13.332856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65be7-368a-4099-b475-048f2e82a7b4 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.339157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1093/2175] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:13.343307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.345559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.346193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.346504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.346829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.347476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.350212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.360958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b11a4-34df-4ea3-ba19-55b386cce2cb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.376633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.377410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.377738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.378012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.380814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:13.384386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.388712 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.409710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1094/2176] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:13.416931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.423677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1083/2177] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:13.428838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.429870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd25ad-c974-4f01-9993-f0f2ef75879a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.447677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:13.520293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.521814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.525622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.525622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.525622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.525923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.539298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3708544-4f35-4fe9-b7a4-b3d89c51c1d2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.544887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.545955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.546116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.546961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.547681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.549932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.566741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1095/2178] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:13.570314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.574805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1084/2179] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:13.581716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.594342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a225-78b7-4d33-aa61-ffc7e6d5e304 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.601736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:13.663976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0f1ac-a4cb-42b3-bb82-d739d836c4a7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.672661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.673313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.673590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.673952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.674547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.676558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.677770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.678458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.678838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.679189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.679950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.681696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.694146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1085/2180] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:13.698517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.702247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1096/2181] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:13.706125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.710585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33314-3c8d-4d27-bfdd-63079ea9109c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.715794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.716412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.716717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.716999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.717646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.719391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.720858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b6c34-4d4f-4020-a4f5-e926d296a396 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.727909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.728771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.729274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.729617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.730328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.731946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1086/2182] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:13.732190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.735806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.745024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1097/2183] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:13.746529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd0470-2a49-439e-92f3-c91e9d87a4a6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.748233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.753243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c8b5d-44d0-48d8-90b7-2d289d2ecba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.754166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8b5d-44d0-48d8-90b7-2d289d2ecba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.754488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8b5d-44d0-48d8-90b7-2d289d2ecba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.754806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c8b5d-44d0-48d8-90b7-2d289d2ecba8 admin admin] PATH_INFO: `/v3/projects/d29c48e27d38433e9927749bcd71ecc1/users/5fbee2d30d75489db903219e6d9e7226/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.759604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a519a0-c6e7-4847-b641-b86d62e7e854 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.764515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.765366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.765488 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.765674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.766483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.768122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.770974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c8b5d-44d0-48d8-90b7-2d289d2ecba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29c48e27d38433e9927749bcd71ecc1, user_id=5fbee2d30d75489db903219e6d9e7226, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.774426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c8b5d-44d0-48d8-90b7-2d289d2ecba8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.779912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1098/2184] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:13.784261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.796574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1087/2185] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:13 2022] PUT /identity/v3/projects/d29c48e27d38433e9927749bcd71ecc1/users/5fbee2d30d75489db903219e6d9e7226/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:13.797436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c356c7-c1bb-4cb1-822a-cbb54c22002e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.806696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721587ba-e60c-422a-b703-479b32f8884b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.806901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721587ba-e60c-422a-b703-479b32f8884b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.807174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721587ba-e60c-422a-b703-479b32f8884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.853161 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-721587ba-e60c-422a-b703-479b32f8884b None None] MFA Rules not processed for user `5fbee2d30d75489db903219e6d9e7226`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:13.856128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-721587ba-e60c-422a-b703-479b32f8884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:13.888747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a32697-cd1a-4909-8652-6eeba8850bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:13.888747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a32697-cd1a-4909-8652-6eeba8850bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.888747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a32697-cd1a-4909-8652-6eeba8850bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.888747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a32697-cd1a-4909-8652-6eeba8850bc0 admin admin] PATH_INFO: `/v3/projects/76b051e429d94764a363f42e35e77428/users/a8d65fec01aa499a92bcc96b83c36d94/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.905583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a32697-cd1a-4909-8652-6eeba8850bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b051e429d94764a363f42e35e77428, user_id=a8d65fec01aa499a92bcc96b83c36d94, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:13.908612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a32697-cd1a-4909-8652-6eeba8850bc0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:13.927040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:13.929400 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1099/2186] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:13 2022] PUT /identity/v3/projects/76b051e429d94764a363f42e35e77428/users/a8d65fec01aa499a92bcc96b83c36d94/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:13.940306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:13.940306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:13.940838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:13.965317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1088/2187] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:13.970013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721587ba-e60c-422a-b703-479b32f8884b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:13.980832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] MFA Rules not processed for user `a8d65fec01aa499a92bcc96b83c36d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:13.982291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721587ba-e60c-422a-b703-479b32f8884b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:13.982722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:14.070509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.071504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.071851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.072142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.072974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.075100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.104536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1100/2188] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:14.113866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.116505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:14.126979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:14.132995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1089/2189] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:13 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:14.134187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.199940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6b50-4cd7-4c23-977b-87bda190cb73 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.214621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c513fe-fb3b-4d79-a4c7-f77ec1eb6163 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.219340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.219939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.220215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.220567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.221618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.223640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.249895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1090/2190] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:14 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:14.253643 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.261508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.262298 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.262768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.263083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.268744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90961501-e1e0-4e94-a7e3-575999ea1ac6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.276290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.277151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.277434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.277710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.278365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.283466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.296702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1091/2191] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:14.301941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.317130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3060d3-452b-4842-a905-8cc595e10e6d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.323365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.324200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.324424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.324807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] PATH_INFO: `/v3/projects/fd313c6cc6724f92805188ff61ec6620/users/fbc2d587a07f49a9822b5f02faeb2f58/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.342211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd313c6cc6724f92805188ff61ec6620, user_id=fbc2d587a07f49a9822b5f02faeb2f58, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.342436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.343487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.344800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.356876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1101/2192] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:14.360599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1092/2193] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:14 2022] PUT /identity/v3/projects/fd313c6cc6724f92805188ff61ec6620/users/fbc2d587a07f49a9822b5f02faeb2f58/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:14.364908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.385417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c21a-9279-4bad-a0f3-67e4ac60e8b0 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.459837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80e9f5-38a2-4214-9fdb-720c50babf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.460458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80e9f5-38a2-4214-9fdb-720c50babf57 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.460773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80e9f5-38a2-4214-9fdb-720c50babf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.461052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80e9f5-38a2-4214-9fdb-720c50babf57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.461756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80e9f5-38a2-4214-9fdb-720c50babf57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.463745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80e9f5-38a2-4214-9fdb-720c50babf57 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.476392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1102/2194] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:14.480179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.503426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a3ced-fd7d-49a8-8646-ceffa4dd2d95 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.509482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.510137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.510414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.510694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] PATH_INFO: `/v3/projects/fd313c6cc6724f92805188ff61ec6620/users/fbc2d587a07f49a9822b5f02faeb2f58/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.512632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd313c6cc6724f92805188ff61ec6620, user_id=fbc2d587a07f49a9822b5f02faeb2f58, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.515460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.527329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1093/2195] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:14 2022] PUT /identity/v3/projects/fd313c6cc6724f92805188ff61ec6620/users/fbc2d587a07f49a9822b5f02faeb2f58/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:14.538873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.539163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.539439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.559811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:14.569441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] MFA Rules not processed for user `fbc2d587a07f49a9822b5f02faeb2f58`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:14.570650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:14.650015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1103/2196] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:14.661124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:14.683114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:14.760128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846af7d-7ad1-4952-a85c-837b5939b652 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:14.765371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:14.766184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:14.766821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:14.767177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:14.853760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:14.855445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:14.866509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1094/2197] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:20.184446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:20.194472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571d80e-97ed-4ac7-bdf6-b59ee896309d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:20.198318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dde427-1c47-4685-91df-1f33be26c9b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:20.198853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dde427-1c47-4685-91df-1f33be26c9b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:20.199129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dde427-1c47-4685-91df-1f33be26c9b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:20.199402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dde427-1c47-4685-91df-1f33be26c9b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:20.213078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dde427-1c47-4685-91df-1f33be26c9b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:20.214527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dde427-1c47-4685-91df-1f33be26c9b4 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:20.224005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1104/2198] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:24.599399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:24.668177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd625e-5635-46d3-95d6-ba6b7ca068db service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:24.673557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:24.673557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:24.673780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:24.673959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] PATH_INFO: `/v3/users/a8d65fec01aa499a92bcc96b83c36d94` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:24.675546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d65fec01aa499a92bcc96b83c36d94)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:24.677670 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:24.732147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d65fec01aa499a92bcc96b83c36d94'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:24.748835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d65fec01aa499a92bcc96b83c36d94'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:24.748835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d65fec01aa499a92bcc96b83c36d94'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:24.757178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d65fec01aa499a92bcc96b83c36d94'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:24.758366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:24.766760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d65fec01aa499a92bcc96b83c36d94'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:24.773570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1095/2199] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:24 2022] DELETE /identity/v3/users/a8d65fec01aa499a92bcc96b83c36d94 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:24.778618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:24.779080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:24.779311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:24.798317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:24.799634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:24.865734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1105/2200] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:24.875775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:24.877305 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:24.943344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef1a65-d4e2-411d-9e71-1f7721a263ba admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:24.947875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:24.948486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:24.948906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:24.949200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:24.959317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:24.961011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:24.972193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1096/2201] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:25.259052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:25.261240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:25.287022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262beed-c53a-4b46-8cb6-8f971bae9b00 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:25.301461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:25.302211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:25.302486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:25.302758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] PATH_INFO: `/v3/projects/76b051e429d94764a363f42e35e77428` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:25.303968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b051e429d94764a363f42e35e77428)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:25.305904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:25.360640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b051e429d94764a363f42e35e77428', 'request_id': 'req-bb983ac9-8668-4865-9dc1-662bac6a857c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.367818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b051e429d94764a363f42e35e77428', 'request_id': 'req-bb983ac9-8668-4865-9dc1-662bac6a857c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.380982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b051e429d94764a363f42e35e77428', 'request_id': 'req-bb983ac9-8668-4865-9dc1-662bac6a857c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.398371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] The token cache is being invalidate because project 76b051e429d94764a363f42e35e77428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:25.398726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.400204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-bb983ac9-8668-4865-9dc1-662bac6a857c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.403521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1106/2202] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:25 2022] DELETE /identity/v3/projects/76b051e429d94764a363f42e35e77428 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:25.446917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:25.448398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:25.465122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f3d80-88d0-4e81-9b77-edf5f6ba4c60 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:25.562723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:25.563378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:25.563656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:25.563933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] PATH_INFO: `/v3/users/5fbee2d30d75489db903219e6d9e7226` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:25.565183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fbee2d30d75489db903219e6d9e7226)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:25.567186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:25.638652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fbee2d30d75489db903219e6d9e7226'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.647204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fbee2d30d75489db903219e6d9e7226'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.647825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fbee2d30d75489db903219e6d9e7226'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.660274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fbee2d30d75489db903219e6d9e7226'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.667502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fbee2d30d75489db903219e6d9e7226'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:25.673510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1097/2203] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:25 2022] DELETE /identity/v3/users/5fbee2d30d75489db903219e6d9e7226 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:25.678618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:25.678914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:25.679194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:25.687640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:25.700568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:25.702357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:25.763936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1107/2204] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:25.773584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:25.775109 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:25.791389 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:25.843701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab997af-1de4-4b1c-b672-5555cb155682 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:25.847598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:25.848206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:25.848482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:25.849329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:25.859940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:25.861528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:25.873203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1098/2205] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:26.232514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:26.234020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:26.261366 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b4e21-ee74-4655-ba6d-91a2a79694c7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:26.276504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:26.277810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:26.279378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:26.279666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] PATH_INFO: `/v3/projects/d29c48e27d38433e9927749bcd71ecc1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:26.282964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29c48e27d38433e9927749bcd71ecc1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:26.285151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:26.348466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29c48e27d38433e9927749bcd71ecc1', 'request_id': 'req-f12e5475-1953-44e7-b47b-2c332166ce8d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.358633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29c48e27d38433e9927749bcd71ecc1', 'request_id': 'req-f12e5475-1953-44e7-b47b-2c332166ce8d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.367572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29c48e27d38433e9927749bcd71ecc1', 'request_id': 'req-f12e5475-1953-44e7-b47b-2c332166ce8d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.380970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] The token cache is being invalidate because project d29c48e27d38433e9927749bcd71ecc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:26.381265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.381932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.383470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1108/2206] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:26 2022] DELETE /identity/v3/projects/d29c48e27d38433e9927749bcd71ecc1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:26.387868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:26.399405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9451-08fa-4e71-b975-1a50a92ea35b service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:26.470241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:26.470910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:26.471196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:26.471469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] PATH_INFO: `/v3/users/fbc2d587a07f49a9822b5f02faeb2f58` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:26.472607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc2d587a07f49a9822b5f02faeb2f58)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:26.475340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:26.532133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc2d587a07f49a9822b5f02faeb2f58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.542244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc2d587a07f49a9822b5f02faeb2f58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.543499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc2d587a07f49a9822b5f02faeb2f58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.561725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc2d587a07f49a9822b5f02faeb2f58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.573039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc2d587a07f49a9822b5f02faeb2f58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:26.584828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1099/2207] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:26 2022] DELETE /identity/v3/users/fbc2d587a07f49a9822b5f02faeb2f58 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:27.045579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.047012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.061280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e5475-1953-44e7-b47b-2c332166ce8d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.062589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.065417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.112206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:27.143931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.144565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.144869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.145141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] PATH_INFO: `/v3/users/1fa4420540af41e4967b0f731f818af5` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.147029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa4420540af41e4967b0f731f818af5)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.147792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de529d2-b415-492b-b87d-e85c36680008 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.149093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.153713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.154572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.155053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.155487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] PATH_INFO: `/v3/projects/fd313c6cc6724f92805188ff61ec6620` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.156666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd313c6cc6724f92805188ff61ec6620)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.158986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.190531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:27.209888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa4420540af41e4967b0f731f818af5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.218708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa4420540af41e4967b0f731f818af5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.219459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa4420540af41e4967b0f731f818af5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.222329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd313c6cc6724f92805188ff61ec6620', 'request_id': 'req-31869abd-24b7-4a66-86f0-8e8265ec0504'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.226949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa4420540af41e4967b0f731f818af5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.228277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd313c6cc6724f92805188ff61ec6620', 'request_id': 'req-31869abd-24b7-4a66-86f0-8e8265ec0504'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.234615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd313c6cc6724f92805188ff61ec6620', 'request_id': 'req-31869abd-24b7-4a66-86f0-8e8265ec0504'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.240139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7b27200b-0f7a-4363-a80b-3b80a239a602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa4420540af41e4967b0f731f818af5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.247138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1109/2208] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:27 2022] DELETE /identity/v3/users/1fa4420540af41e4967b0f731f818af5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:27.247268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] The token cache is being invalidate because project fd313c6cc6724f92805188ff61ec6620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:27.247401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.248103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-31869abd-24b7-4a66-86f0-8e8265ec0504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.249624 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1100/2209] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:27 2022] DELETE /identity/v3/projects/fd313c6cc6724f92805188ff61ec6620 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:27.256166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.256621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.256936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.279546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:27.280807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.342723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1110/2210] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:27.351229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.351511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.351785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.371065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:27.372308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.381096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1101/2211] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:27.385614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.396392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb3c26-4261-4970-9a72-7a0f7daf86af None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.401480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.402101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.402415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.402699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.403379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:27.403676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.405317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.417426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1111/2212] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:27.422023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.427159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.428940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.432279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f151-580d-4c20-a5bd-2d9538cad791 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.440594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.441290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.441633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.442054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.443272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.445255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb80aa0-6aec-4475-a58b-69be608c3818 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.445512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.450492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.451717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.452701 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.453335 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] PATH_INFO: `/v3/projects/43d4440dd2464bcb95e25ead37c5658a` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.454505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d4440dd2464bcb95e25ead37c5658a)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.456421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.476586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1102/2213] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:27.480846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.507023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d4440dd2464bcb95e25ead37c5658a', 'request_id': 'req-5ab98dd3-555f-491a-b36b-1b46571d6c1a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.514071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d4440dd2464bcb95e25ead37c5658a', 'request_id': 'req-5ab98dd3-555f-491a-b36b-1b46571d6c1a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.523115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d4440dd2464bcb95e25ead37c5658a', 'request_id': 'req-5ab98dd3-555f-491a-b36b-1b46571d6c1a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.533645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] The token cache is being invalidate because project 43d4440dd2464bcb95e25ead37c5658a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:27.533929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.534564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5ab98dd3-555f-491a-b36b-1b46571d6c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:27.536752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1112/2214] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:27 2022] DELETE /identity/v3/projects/43d4440dd2464bcb95e25ead37c5658a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:06:27.541449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953d658-6f86-40fb-a940-1529acb0909e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.541728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953d658-6f86-40fb-a940-1529acb0909e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.542003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953d658-6f86-40fb-a940-1529acb0909e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.544638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e4e79-7840-449f-a9b1-503dad2ace9c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.561137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3953d658-6f86-40fb-a940-1529acb0909e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:27.562330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3953d658-6f86-40fb-a940-1529acb0909e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.575630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:27.587343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:27.609509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.610218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.610382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.610793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.611437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.613413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.624741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1113/2215] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:27.632384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.632696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.632970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.634879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1103/2216] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:27.638723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.640325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.652561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebeea1-fe59-4e89-b8ce-d6d8934ecba1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.652871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:27.653826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.657235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.657816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.658093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.658357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.658982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.660570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.662634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1114/2217] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:27.666887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.674078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1104/2218] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:27.675019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.677200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aba6ec-5341-493f-b095-7b5e2891607b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.682054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.682841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.683122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.683397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.684435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:27.684605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.685893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34716124-b317-409c-8314-40812ac2bfbc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.686626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.690679 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.691284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.691561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.691831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.692611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.694251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.699687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1115/2219] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:06:27.703955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.710359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1105/2220] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:27.711335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.715950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bf0b8-5d1c-4458-8922-12107f677ada admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.720900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.721379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409a731-dacd-473c-a9ae-3fc3d1bad50f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.721636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.722011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.722300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.722997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.725737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.727132 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c90be0-938a-42fe-a323-66c34791ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.727894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c90be0-938a-42fe-a323-66c34791ff22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.728299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c90be0-938a-42fe-a323-66c34791ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.728578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c90be0-938a-42fe-a323-66c34791ff22 admin admin] PATH_INFO: `/v3/projects/5cfe017b05264ffb8a03f427a29212d3/users/02377f3a47574ca68e07e9dd0e478b97/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.751753 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c90be0-938a-42fe-a323-66c34791ff22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cfe017b05264ffb8a03f427a29212d3, user_id=02377f3a47574ca68e07e9dd0e478b97, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.756107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c90be0-938a-42fe-a323-66c34791ff22 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.762403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1116/2221] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:06:27.768667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.772201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1106/2222] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:27 2022] PUT /identity/v3/projects/5cfe017b05264ffb8a03f427a29212d3/users/02377f3a47574ca68e07e9dd0e478b97/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:27.784130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.784415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.784724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.819711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] MFA Rules not processed for user `02377f3a47574ca68e07e9dd0e478b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:27.821299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.841397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e50479-3903-4033-a1b9-5098ba7be9fb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.889399 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1107/2223] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:27.900009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.900735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.901086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.901370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.901479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.902031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.902756 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:27.903910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.927122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1117/2224] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:27 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:27.930994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.940935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cf539-6852-4f14-8fab-1aac4d9b01d6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.946410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.947124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.947405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.947676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.948310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.949955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.963194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1118/2225] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:06:27.963352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74df045-b4f6-4ad5-8aa6-93842e3c766c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.964246 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:27.965504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.966107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.966386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.966654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.969029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:27.975204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32912eb-f3e8-48da-ae15-f82875e5503d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:27.978173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.979821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.981016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab727c8-e8be-44e4-a5f5-8a2d026b2464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:27.981624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab727c8-e8be-44e4-a5f5-8a2d026b2464 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:27.981903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab727c8-e8be-44e4-a5f5-8a2d026b2464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:27.982197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab727c8-e8be-44e4-a5f5-8a2d026b2464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:27.982824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab727c8-e8be-44e4-a5f5-8a2d026b2464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:27.984391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab727c8-e8be-44e4-a5f5-8a2d026b2464 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:27.989186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1108/2226] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:27.999966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1119/2227] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:28.001238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:28.012313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140c56a-ba04-42e0-a83e-68b87e0c9cf2 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:28.017014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:28.017659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:28.017936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:28.018205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] PATH_INFO: `/v3/projects/c1913d0c899c48ab9a7781a9e45ac4a9/users/1dc5f1d34cad437ea15efb98a0a1ca31/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:28.033887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1913d0c899c48ab9a7781a9e45ac4a9, user_id=1dc5f1d34cad437ea15efb98a0a1ca31, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:28.036571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:28.039067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:28.051696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1109/2228] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:28 2022] PUT /identity/v3/projects/c1913d0c899c48ab9a7781a9e45ac4a9/users/1dc5f1d34cad437ea15efb98a0a1ca31/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:28.057225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46375f-e429-41dd-bfb0-5892f45161c2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:28.057538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46375f-e429-41dd-bfb0-5892f45161c2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:28.057816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46375f-e429-41dd-bfb0-5892f45161c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:28.079408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ed46375f-e429-41dd-bfb0-5892f45161c2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:28.080591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ed46375f-e429-41dd-bfb0-5892f45161c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:28.145867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1120/2229] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:28.155577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:28.157378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:28.225526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b011-7d34-4b5d-a46e-91ecd031af79 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:28.229167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:28.229774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:28.230048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:28.230316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:28.244330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:28.245803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:28.256368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1110/2230] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:32.438703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fe0bc-bc83-4e20-bbe0-e209f893b213 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.439198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fe0bc-bc83-4e20-bbe0-e209f893b213 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.439290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fe0bc-bc83-4e20-bbe0-e209f893b213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.468961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-273fe0bc-bc83-4e20-bbe0-e209f893b213 None None] MFA Rules not processed for user `1dc5f1d34cad437ea15efb98a0a1ca31`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:32.470055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-273fe0bc-bc83-4e20-bbe0-e209f893b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:32.537002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1121/2231] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:32.550873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:32.552951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:32.631301 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29097d-66e2-4e8a-bcee-a17f8c126d98 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:32.636921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:32.637525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:32.637802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:32.638076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:32.646954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:32.648562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:32.658660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1111/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:34.261955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849e8b4-4f45-4e60-9578-227071cce787 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:34.262451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849e8b4-4f45-4e60-9578-227071cce787 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:34.262545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849e8b4-4f45-4e60-9578-227071cce787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:34.282553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d849e8b4-4f45-4e60-9578-227071cce787 None None] MFA Rules not processed for user `1dc5f1d34cad437ea15efb98a0a1ca31`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:34.283738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d849e8b4-4f45-4e60-9578-227071cce787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:34.290985 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1122/2233] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:42.655901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:INSERT=1,keystone:SELECT=62 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:44.210965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.224506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c415855-e381-47c2-9574-daa56e42b0c8 service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.229456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.230220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.230597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.230968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] PATH_INFO: `/v3/users/02377f3a47574ca68e07e9dd0e478b97` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.232254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02377f3a47574ca68e07e9dd0e478b97)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.234932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.276611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=66 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:44.304135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02377f3a47574ca68e07e9dd0e478b97'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:44.312687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02377f3a47574ca68e07e9dd0e478b97'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:44.314120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02377f3a47574ca68e07e9dd0e478b97'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:44.327020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02377f3a47574ca68e07e9dd0e478b97'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:44.334714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02377f3a47574ca68e07e9dd0e478b97'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:44.341366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1112/2234] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:44 2022] DELETE /identity/v3/users/02377f3a47574ca68e07e9dd0e478b97 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:44.345888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.346167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.346437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.365772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:44.366954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:44.426785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1123/2235] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:44.436592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:44.438043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:44.496368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf64d2-f563-4144-bebe-90b1729a3d02 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:44.499943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220e8fc-5d4b-44a5-8d27-3f4d71332f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:44.500532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220e8fc-5d4b-44a5-8d27-3f4d71332f33 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:44.500852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220e8fc-5d4b-44a5-8d27-3f4d71332f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:44.501217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220e8fc-5d4b-44a5-8d27-3f4d71332f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:44.511247 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220e8fc-5d4b-44a5-8d27-3f4d71332f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:44.512739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220e8fc-5d4b-44a5-8d27-3f4d71332f33 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:44.522189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1113/2236] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:45.014413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.034992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:45.065594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d844a-b675-4f73-b318-37b0914529f9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.073977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.074700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.075046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.075402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] PATH_INFO: `/v3/projects/5cfe017b05264ffb8a03f427a29212d3` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.076743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cfe017b05264ffb8a03f427a29212d3)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.078747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.132483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cfe017b05264ffb8a03f427a29212d3', 'request_id': 'req-26819e94-2bad-4c8f-a870-8b1561c92dfe'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.139513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cfe017b05264ffb8a03f427a29212d3', 'request_id': 'req-26819e94-2bad-4c8f-a870-8b1561c92dfe'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.148878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cfe017b05264ffb8a03f427a29212d3', 'request_id': 'req-26819e94-2bad-4c8f-a870-8b1561c92dfe'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.159590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] The token cache is being invalidate because project 5cfe017b05264ffb8a03f427a29212d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:45.159877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.161116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26819e94-2bad-4c8f-a870-8b1561c92dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:45.162640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1124/2237] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:45 2022] DELETE /identity/v3/projects/5cfe017b05264ffb8a03f427a29212d3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:45.167626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.167906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.168182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.187797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:45.189074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:45.251728 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1114/2238] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:45.260130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.260489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.261017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.282207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:45.283398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:45.291445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1125/2239] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:45.295751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.305612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafba88-6349-4038-8382-3508593e2d24 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.319186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.319810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.320083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.320350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.321296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:06:45.321596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.323207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.335785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1115/2240] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:06:45.340291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.351078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f55f65-73a9-459a-9b21-11a486b893c4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.356137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.356769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.357055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.357332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.358045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.360021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.389112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1126/2241] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:06:45 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:45.393537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.398645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:45.404964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca1f4c-9fe6-4544-860e-628c4bb6a5cc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.462002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.462601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.462877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.463149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.463774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.465732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.488246 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1116/2242] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:06:45 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:06:45.493319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.504058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b68481-fa28-458b-b4c8-04057cb5d879 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.508763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.509351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.509636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.509908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.510572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.512189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.522134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1127/2243] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:45.526001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.536302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a1aa6-1fff-4ebc-a15a-0bd753ebda6b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.540948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781de42e-3385-4c54-b30f-126f3d776631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.541538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781de42e-3385-4c54-b30f-126f3d776631 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.541811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781de42e-3385-4c54-b30f-126f3d776631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.542082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781de42e-3385-4c54-b30f-126f3d776631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.542718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781de42e-3385-4c54-b30f-126f3d776631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.544302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781de42e-3385-4c54-b30f-126f3d776631 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.555300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1117/2244] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:06:45.559143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.568979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ca31b-b1be-4744-a493-f381c4957b20 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.573865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.574503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.574785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.575062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] PATH_INFO: `/v3/projects/68d5f346b3cd47209f3719aa2887017d/users/6d4a1d878c9946e1b6c1ca341d3337b1/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.590185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d5f346b3cd47209f3719aa2887017d, user_id=6d4a1d878c9946e1b6c1ca341d3337b1, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.592922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.608229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1128/2245] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:06:45 2022] PUT /identity/v3/projects/68d5f346b3cd47209f3719aa2887017d/users/6d4a1d878c9946e1b6c1ca341d3337b1/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:45.619323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.619609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.619886 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.647704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] MFA Rules not processed for user `6d4a1d878c9946e1b6c1ca341d3337b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:45.648946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:45.714169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1118/2246] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:06:45.725614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:45.727351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:45.749568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:45.787086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8c0c2-207c-4388-b0fb-1d9a44c5fce9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:45.790751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2157480-495c-4e42-9e73-0e2934724876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:45.791345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157480-495c-4e42-9e73-0e2934724876 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:45.791626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157480-495c-4e42-9e73-0e2934724876 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:45.791888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157480-495c-4e42-9e73-0e2934724876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:45.799917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157480-495c-4e42-9e73-0e2934724876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:45.801887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157480-495c-4e42-9e73-0e2934724876 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:45.810756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1129/2247] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:06:49.185406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:49.264551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344d522-1e53-4028-bfb1-ae78c5449fd4 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:49.269640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:49.270406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:49.270718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:49.271020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] PATH_INFO: `/v3/users/6d4a1d878c9946e1b6c1ca341d3337b1` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:49.272139 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d4a1d878c9946e1b6c1ca341d3337b1)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:49.274136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:49.288023 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:49.324277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d4a1d878c9946e1b6c1ca341d3337b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.332056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d4a1d878c9946e1b6c1ca341d3337b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.332915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d4a1d878c9946e1b6c1ca341d3337b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.345159 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d4a1d878c9946e1b6c1ca341d3337b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.353398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d4a1d878c9946e1b6c1ca341d3337b1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.359347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1119/2248] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:06:49 2022] DELETE /identity/v3/users/6d4a1d878c9946e1b6c1ca341d3337b1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:49.364918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:49.365468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:49.365468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:49.384979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:06:49.386174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:49.446006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1130/2249] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:06:49.455632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:49.457655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:49.514525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a3b14-cbdf-47f0-9903-2df997ff33c2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:49.518088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:49.518675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:49.518948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:49.519218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:49.528426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:49.529881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:49.539848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1120/2250] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:06:49.804574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:06:49.806070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:06:49.820961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424233a5-b808-4bc9-a4a5-007dc06edb08 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:06:49.826038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:06:49.826669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:06:49.826947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:06:49.833241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] PATH_INFO: `/v3/projects/68d5f346b3cd47209f3719aa2887017d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:06:49.834984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d5f346b3cd47209f3719aa2887017d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:06:49.837005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:06:49.892280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d5f346b3cd47209f3719aa2887017d', 'request_id': 'req-77c007fe-b416-480f-8e35-b85270b34b59'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.899799 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d5f346b3cd47209f3719aa2887017d', 'request_id': 'req-77c007fe-b416-480f-8e35-b85270b34b59'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.909352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d5f346b3cd47209f3719aa2887017d', 'request_id': 'req-77c007fe-b416-480f-8e35-b85270b34b59'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.921475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] The token cache is being invalidate because project 68d5f346b3cd47209f3719aa2887017d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:06:49.921767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.922418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-77c007fe-b416-480f-8e35-b85270b34b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:06:49.924552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1131/2251] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:06:49 2022] DELETE /identity/v3/projects/68d5f346b3cd47209f3719aa2887017d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:06:59.535722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:59.918293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:00.155285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:00.156760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:00.172986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082522ed-9320-4b83-99f0-0db50279aff7 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:00.242189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:00.242835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.243114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.243398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] PATH_INFO: `/v3/users/61b2ea9567664c60bda3d3d06b89dccd` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.244544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b2ea9567664c60bda3d3d06b89dccd)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:00.246903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:00.302225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b2ea9567664c60bda3d3d06b89dccd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:00.315047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b2ea9567664c60bda3d3d06b89dccd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:00.315677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b2ea9567664c60bda3d3d06b89dccd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:00.330345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b2ea9567664c60bda3d3d06b89dccd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:00.338712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b2ea9567664c60bda3d3d06b89dccd'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:00.346196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1121/2252] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:00 2022] DELETE /identity/v3/users/61b2ea9567664c60bda3d3d06b89dccd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:00.351249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.352285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.352872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.376639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:00.377964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:00.454954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1132/2253] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:00.462045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:00.463490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:00.525678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac28dd-f28d-40bb-a836-111b9156b6de admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:00.529377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3571a9f-e79a-43c2-98f8-a8e51fc6faad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:00.529979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3571a9f-e79a-43c2-98f8-a8e51fc6faad service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.530257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3571a9f-e79a-43c2-98f8-a8e51fc6faad service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.530529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3571a9f-e79a-43c2-98f8-a8e51fc6faad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.540226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3571a9f-e79a-43c2-98f8-a8e51fc6faad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:00.541704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3571a9f-e79a-43c2-98f8-a8e51fc6faad service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:00.551971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1122/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:00.894806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:00.896899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:00.910927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0a65d-741a-4693-8e75-461463079ff5 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:00.916144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:00.917114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:00.917860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:00.917860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] PATH_INFO: `/v3/projects/64c2866bc8cd49348487702c71865a1a` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:00.918990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c2866bc8cd49348487702c71865a1a)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:00.921166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:00.988310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c2866bc8cd49348487702c71865a1a', 'request_id': 'req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:00.994560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c2866bc8cd49348487702c71865a1a', 'request_id': 'req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:01.002155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c2866bc8cd49348487702c71865a1a', 'request_id': 'req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:01.014716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] The token cache is being invalidate because project 64c2866bc8cd49348487702c71865a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:01.015013 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:01.015705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8ec0f2ae-3cb7-4f97-bf0c-326dcea60e45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:01.017442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1133/2255] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:00 2022] DELETE /identity/v3/projects/64c2866bc8cd49348487702c71865a1a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:01.023955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.024235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.024533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.044499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:01.045737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:01.064738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:01.116106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1123/2256] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:01.124857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.125144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.125426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.146454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:01.147650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:01.155468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1134/2257] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:01.159864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.169701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156000c-dd8d-40a1-9460-9abfa9f2488c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.174200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.174787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.175067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.175435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.176253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:01.176558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.178226 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.190350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1124/2258] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:01.194286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.203964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ec29a-06dd-4c9d-88a1-0a325530ff77 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.210680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.211288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.211570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.211849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.212490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.214440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.240760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1135/2259] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:01.244919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.255292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d472f-d8b0-4d73-978b-33db255e5ffd admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.310420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.311012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.311287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.311553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.312178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.314110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.335409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1125/2260] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:01.339513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.349832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1fcb-6909-48c2-bded-6ec173b1f1c0 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.354490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.355078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.355353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.355623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.356254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.357863 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.368283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1136/2261] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:01.372112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.382522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fef32-1cf3-4c1f-b710-958f89b923ed admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.387083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6a44c-a910-4289-9166-82f922839139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.387674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6a44c-a910-4289-9166-82f922839139 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.387952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6a44c-a910-4289-9166-82f922839139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.388224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6a44c-a910-4289-9166-82f922839139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.388891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6a44c-a910-4289-9166-82f922839139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.390493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6a44c-a910-4289-9166-82f922839139 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.401239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1126/2262] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:01.405379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.415201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03858dd6-acaf-4d0b-b4b3-c303c1a35cee admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.419797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.420378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.420652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.420950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] PATH_INFO: `/v3/projects/7b9af3a8a7b3493a817ec028e67f707f/users/9c13e37200d0432788156ae78daaca53/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.435446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9af3a8a7b3493a817ec028e67f707f, user_id=9c13e37200d0432788156ae78daaca53, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.438131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.451834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1137/2263] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:01 2022] PUT /identity/v3/projects/7b9af3a8a7b3493a817ec028e67f707f/users/9c13e37200d0432788156ae78daaca53/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:07:01.462140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca0c10-6ee6-4c7d-96c5-0967947e6815 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.462445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca0c10-6ee6-4c7d-96c5-0967947e6815 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.462716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca0c10-6ee6-4c7d-96c5-0967947e6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.490028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-fcca0c10-6ee6-4c7d-96c5-0967947e6815 None None] MFA Rules not processed for user `9c13e37200d0432788156ae78daaca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:01.491210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fcca0c10-6ee6-4c7d-96c5-0967947e6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:01.553435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1127/2264] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:01.564066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:01.565763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:01.587904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:01.624401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38110d0c-de07-44f9-9367-fd6d1ec1486e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:01.628019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05a995-5601-4c95-a092-24025bc248e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:01.629443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05a995-5601-4c95-a092-24025bc248e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:01.629725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05a995-5601-4c95-a092-24025bc248e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:01.630000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05a995-5601-4c95-a092-24025bc248e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:01.638067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05a995-5601-4c95-a092-24025bc248e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:01.639482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05a995-5601-4c95-a092-24025bc248e9 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:01.648056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1138/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:03.002442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:03.002913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:03.003005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:03.022210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] MFA Rules not processed for user `9c13e37200d0432788156ae78daaca53`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:03.023407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:03.030220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1128/2266] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:11.644170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=17 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:13.016276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:15.275434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b5bc2-f3c2-4294-bc73-2c2e0d1d942e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:15.275936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b5bc2-f3c2-4294-bc73-2c2e0d1d942e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:15.276119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b5bc2-f3c2-4294-bc73-2c2e0d1d942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:15.296473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-027b5bc2-f3c2-4294-bc73-2c2e0d1d942e None None] MFA Rules not processed for user `a3dac44116784a9e963a69e4793bac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:15.298027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-027b5bc2-f3c2-4294-bc73-2c2e0d1d942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:15.352468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1139/2267] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:25.039997 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:25.041584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:25.117344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191f1a7-1947-4a18-abd7-89a2f00b1c17 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:25.122520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:25.123340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:25.123617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:25.123892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:25.125671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:25.137478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:25.139067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:25.149065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1129/2268] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:25.345927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=25 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:27.316512 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551087f-5bb2-4c3e-96f2-082153469ce5 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.317136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551087f-5bb2-4c3e-96f2-082153469ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.317136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7551087f-5bb2-4c3e-96f2-082153469ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.337022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7551087f-5bb2-4c3e-96f2-082153469ce5 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:27.338223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7551087f-5bb2-4c3e-96f2-082153469ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:27.392162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1140/2269] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:27.432898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:27.434354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:27.501862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821fd96-d684-43a6-a7a8-1c3dfe2e5a23 service placement] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:27.507587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:27.508177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:27.508454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:27.508755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:27.516712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:27.519003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:27.526805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1130/2270] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:28.109332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed869-bdbf-4d00-84d1-4fce731be701 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.109613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed869-bdbf-4d00-84d1-4fce731be701 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.109881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ed869-bdbf-4d00-84d1-4fce731be701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.131948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2e2ed869-bdbf-4d00-84d1-4fce731be701 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:28.133320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2e2ed869-bdbf-4d00-84d1-4fce731be701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:28.140475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1141/2271] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 17 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:28.153258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.164075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec3c3b-57a8-4b0d-8a0f-4869cbc71c75 service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.169199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.169785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.170062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.170349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.179341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.180992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.189084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1131/2272] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:28.427007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427811d3-f5ae-4537-9cbb-1feda5a9ba88 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.427492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427811d3-f5ae-4537-9cbb-1feda5a9ba88 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.427609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427811d3-f5ae-4537-9cbb-1feda5a9ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.447758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-427811d3-f5ae-4537-9cbb-1feda5a9ba88 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:28.448964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-427811d3-f5ae-4537-9cbb-1feda5a9ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:28.455916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1142/2273] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:28.467941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.478298 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac0809-537a-4971-8745-050c5deac4f5 service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.483319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.483904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.484182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.484457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.492827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.494409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.502362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1132/2274] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:28.847881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.848160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.848455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.867930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:28.869192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:28.876230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1143/2275] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 17 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:28.888373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:28.898654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c7741-2d32-4e4e-9181-bbe559845976 service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:28.903609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e828bb4-a3c2-4149-a546-ca813ebc827d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:28.904188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828bb4-a3c2-4149-a546-ca813ebc827d service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:28.904461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828bb4-a3c2-4149-a546-ca813ebc827d service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:28.904765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e828bb4-a3c2-4149-a546-ca813ebc827d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:28.913382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828bb4-a3c2-4149-a546-ca813ebc827d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:28.915120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e828bb4-a3c2-4149-a546-ca813ebc827d service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:28.923630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1133/2276] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:07:31.021693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:31.083977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca75c1-8cd7-4680-bddf-b9933560815c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:31.088487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:31.089140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.089420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.089692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] PATH_INFO: `/v3/users/9c13e37200d0432788156ae78daaca53` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.090789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c13e37200d0432788156ae78daaca53)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:31.092749 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:31.137900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c13e37200d0432788156ae78daaca53'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.144657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c13e37200d0432788156ae78daaca53'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.145290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c13e37200d0432788156ae78daaca53'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.152129 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c13e37200d0432788156ae78daaca53'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.164132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c13e37200d0432788156ae78daaca53'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.170085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1144/2277] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:07:31 2022] DELETE /identity/v3/users/9c13e37200d0432788156ae78daaca53 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:31.174627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.175080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.175204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.194538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:31.195969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:31.229224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:31.255557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1134/2278] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:31.265198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:31.266592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:31.273014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:31.327921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcdc11-6dad-4df0-b2f4-4658a98c4e53 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:31.331641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e248b-33b5-4ab7-8441-d337bd97d29c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:31.332754 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e248b-33b5-4ab7-8441-d337bd97d29c service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.333029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e248b-33b5-4ab7-8441-d337bd97d29c service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.333357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e248b-33b5-4ab7-8441-d337bd97d29c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.342519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e248b-33b5-4ab7-8441-d337bd97d29c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:31.343933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e248b-33b5-4ab7-8441-d337bd97d29c service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:31.354208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1145/2279] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:31.575802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:31.577678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:31.597016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe7a88-2a78-4c72-9245-818b2b27e374 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:31.602590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:31.603304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.603647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.604034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] PATH_INFO: `/v3/projects/7b9af3a8a7b3493a817ec028e67f707f` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.605242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9af3a8a7b3493a817ec028e67f707f)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:31.607422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:31.660991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9af3a8a7b3493a817ec028e67f707f', 'request_id': 'req-b8578461-36c3-4741-92ac-de47b695d534'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.666661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9af3a8a7b3493a817ec028e67f707f', 'request_id': 'req-b8578461-36c3-4741-92ac-de47b695d534'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.673100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9af3a8a7b3493a817ec028e67f707f', 'request_id': 'req-b8578461-36c3-4741-92ac-de47b695d534'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.686596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] The token cache is being invalidate because project 7b9af3a8a7b3493a817ec028e67f707f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:07:31.686883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.687495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b8578461-36c3-4741-92ac-de47b695d534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:07:31.689008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1135/2280] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:07:31 2022] DELETE /identity/v3/projects/7b9af3a8a7b3493a817ec028e67f707f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:31.694697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.695228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.695512 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.716140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:31.717365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:31.778877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1146/2281] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:31.786957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.787242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.787524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.807915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:31.809162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:31.817584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1136/2282] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:31.822360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:31.831978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021b1bb-9ef8-4c58-81c2-f21c4f6493f3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:31.837531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:31.838132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.838407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.838676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.839318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:07:31.839618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:31.842377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:31.854319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1147/2283] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:07:31.858697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:31.868054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9952-ec93-44b2-b904-b0e88c36959c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:31.872650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:31.873263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.873535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.873805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.874431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:31.876318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:31.909304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1137/2284] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:31 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:31.910537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:31.920964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68578ac-743f-4cd2-9795-c2c95c92453f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:31.980497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:31.981410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:31.981828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:31.982228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:31.983197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:31.986281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:32.014815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1148/2285] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:31 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:32.019496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:32.030764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e68f5-9b15-471a-92a6-ca43994f8ade admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:32.035404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:32.036136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:32.036413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:32.036700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:32.037350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:32.038943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:32.049456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1138/2286] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:07:32.054721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:32.065389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da972fc7-d1b7-48bb-a8fa-7b2fcfa552ad admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:32.071387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0068e5-b25a-4a12-865f-d6232b78418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:32.072058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0068e5-b25a-4a12-865f-d6232b78418f admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:32.072405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0068e5-b25a-4a12-865f-d6232b78418f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:32.072776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0068e5-b25a-4a12-865f-d6232b78418f admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:32.073485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0068e5-b25a-4a12-865f-d6232b78418f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:32.075169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0068e5-b25a-4a12-865f-d6232b78418f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:32.087213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1149/2287] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:32.091567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:32.101823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9c922-cfeb-4b06-b89e-f2a598c08271 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:32.106654 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:32.107302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:32.107578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:32.107851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] PATH_INFO: `/v3/projects/36c930f8134548e393831a0ef4034ea9/users/c829f155714140bb87a78429d4e50af3/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:32.125541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c930f8134548e393831a0ef4034ea9, user_id=c829f155714140bb87a78429d4e50af3, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:32.128200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:32.141902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1139/2288] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:32 2022] PUT /identity/v3/projects/36c930f8134548e393831a0ef4034ea9/users/c829f155714140bb87a78429d4e50af3/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:07:32.147108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998587c-d36b-4520-84d7-8a78555a8919 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:32.147394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998587c-d36b-4520-84d7-8a78555a8919 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:32.147668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5998587c-d36b-4520-84d7-8a78555a8919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:32.166929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5998587c-d36b-4520-84d7-8a78555a8919 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:32.168173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5998587c-d36b-4520-84d7-8a78555a8919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:32.176866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:32.230028 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1150/2289] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:32.240283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:32.242046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:32.299188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755815d-9835-4db8-bcd2-79c57548a8db admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:32.303765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:32.304368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:32.304701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:32.305027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:32.315350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:32.315833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:32.326659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1140/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:36.510877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.511417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.511521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.541691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] MFA Rules not processed for user `c829f155714140bb87a78429d4e50af3`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:36.542899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:36.609570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1151/2291] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:36.614421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.619377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:36.624258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626504f-32ad-4680-8318-05b4724595ee service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.631640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.632437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.633098 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.642852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.642852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.642852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.678893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1141/2292] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:07:36 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:07:36.683017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.696821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01133a-3ecf-4f92-962f-210c916884ed None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.757823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.758435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.758719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.758988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.759618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.761582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.786764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1152/2293] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:07:36 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:07:36.790882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.801946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f52843-48e0-4784-9fce-25fc9b3c5e5a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.806900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.807500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.807773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.808042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.808700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.810310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.821835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1142/2294] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:36.825979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.837053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4a9a-ed5a-4de5-93ab-8367b0672062 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.841692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.842344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.842619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.842938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] PATH_INFO: `/v3/projects/f0feb1aff84b469c9148c81063fcdf88/users/91a118aa4be74fd99e7e3768e439c406/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.860014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0feb1aff84b469c9148c81063fcdf88, user_id=91a118aa4be74fd99e7e3768e439c406, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.862719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.876477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1153/2295] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:36 2022] PUT /identity/v3/projects/f0feb1aff84b469c9148c81063fcdf88/users/91a118aa4be74fd99e7e3768e439c406/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:36.880510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.891867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079cad9-23ab-43f5-bf2f-af72ce49dd6a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.950776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967ee88-46de-4143-b793-3a2f6dee6a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.951442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967ee88-46de-4143-b793-3a2f6dee6a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.951729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967ee88-46de-4143-b793-3a2f6dee6a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.952002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967ee88-46de-4143-b793-3a2f6dee6a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.952646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967ee88-46de-4143-b793-3a2f6dee6a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.954296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967ee88-46de-4143-b793-3a2f6dee6a54 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.966213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1143/2296] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:07:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:07:36.970487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:36.981633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6d526-c18b-46dc-9600-269cfb581728 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:36.986333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:36.987009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:36.987288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:36.987560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] PATH_INFO: `/v3/projects/f0feb1aff84b469c9148c81063fcdf88/users/91a118aa4be74fd99e7e3768e439c406/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:36.989761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0feb1aff84b469c9148c81063fcdf88, user_id=91a118aa4be74fd99e7e3768e439c406, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:36.992449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:36.996209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:37.007963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1154/2297] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:07:36 2022] PUT /identity/v3/projects/f0feb1aff84b469c9148c81063fcdf88/users/91a118aa4be74fd99e7e3768e439c406/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:07:41.234030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad89c-2c28-45bf-8f30-50cf9e9faf58 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:41.234608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad89c-2c28-45bf-8f30-50cf9e9faf58 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:41.234608 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ad89c-2c28-45bf-8f30-50cf9e9faf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:41.264282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-094ad89c-2c28-45bf-8f30-50cf9e9faf58 None None] MFA Rules not processed for user `91a118aa4be74fd99e7e3768e439c406`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:41.265564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-094ad89c-2c28-45bf-8f30-50cf9e9faf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:41.339115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1144/2298] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:07:41.353852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:07:41.355651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:41.440319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042564-2ea9-43e1-86f9-a79433ec4908 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:07:41.446915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1deb-2cb5-4284-ac38-4f20951cefe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:07:41.447522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1deb-2cb5-4284-ac38-4f20951cefe4 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:41.447865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1deb-2cb5-4284-ac38-4f20951cefe4 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:41.448068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1deb-2cb5-4284-ac38-4f20951cefe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:41.505442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b1deb-2cb5-4284-ac38-4f20951cefe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:07:41.507124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b1deb-2cb5-4284-ac38-4f20951cefe4 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:07:41.516360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1155/2299] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:07:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:07:42.594268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:07:42.594729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:07:42.594839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:07:42.614435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] MFA Rules not processed for user `c829f155714140bb87a78429d4e50af3`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:07:42.615639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:07:42.622545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1145/2300] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:07:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:07:51.512054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=58 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:52.608377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:05.993774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bc03a-eec4-440d-b3d1-50553d5bde5d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:05.994258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bc03a-eec4-440d-b3d1-50553d5bde5d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:05.994360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bc03a-eec4-440d-b3d1-50553d5bde5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:06.013713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0d2bc03a-eec4-440d-b3d1-50553d5bde5d None None] MFA Rules not processed for user `1e3b1e0f4888400b89c47b3e44443228`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:08:06.014893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0d2bc03a-eec4-440d-b3d1-50553d5bde5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:08:06.068953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1156/2301] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:08:16.061617 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=25 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:31.803945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:08:31.858569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53e2f-692b-4f6a-9b48-464ed7691724 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:08:31.862056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:08:31.862666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:08:31.862940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:08:31.863206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:08:31.872518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:08:31.874092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:08:31.883350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1146/2302] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:08:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:08:41.879148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=26 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:08.507070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:08.507556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:08.507657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:08.527367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:08.528550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:08.582638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1157/2303] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:08.629153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:08.631449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:08.700306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb3962-5412-4f39-a385-6fa8e9d5c4c5 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:08.705417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:08.705963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:08.706245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:08.706523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:08.715009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:08.716615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:08.726344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1147/2304] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:09:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:09:12.629641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:12.642581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040c64c-f085-4b17-98c4-e59de4875510 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:12.648008 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:12.648707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:12.649089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:12.649542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:12.652234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:12.682225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:12.684099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bd739-cf72-4fa2-b461-5011d1629216 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:12.697634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1158/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:14.454280 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:14.465107 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a10fd-ff95-4881-9bc6-36156d190883 service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:14.469373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:14.469987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:14.470269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:14.470560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:14.472439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:14.488095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:14.489584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50377-9fa6-489c-840b-3e07d53d0441 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:14.507239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1148/2306] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:22.690875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=31 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:23.726955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9d26-6cad-4a0c-aae6-a0dbf5c980bf None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:23.727640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9d26-6cad-4a0c-aae6-a0dbf5c980bf None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:23.728426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d9d26-6cad-4a0c-aae6-a0dbf5c980bf None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:23.730074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1159/2307] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:09:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:09:23.736521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002f0c-083d-4682-8b45-758a67f5b037 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:23.737111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002f0c-083d-4682-8b45-758a67f5b037 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:23.737111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002f0c-083d-4682-8b45-758a67f5b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:23.761459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-3e002f0c-083d-4682-8b45-758a67f5b037 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:23.763121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3e002f0c-083d-4682-8b45-758a67f5b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:23.774931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1149/2308] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:23.787144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d9d26-6cad-4a0c-aae6-a0dbf5c980bf None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:23.789444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-fd3d9d26-6cad-4a0c-aae6-a0dbf5c980bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:23.848506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d9d26-6cad-4a0c-aae6-a0dbf5c980bf None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:23.852790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a82f94-c79a-455d-a4b6-efcc144a9b38 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:23.853397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a82f94-c79a-455d-a4b6-efcc144a9b38 service glance] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:23.853700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a82f94-c79a-455d-a4b6-efcc144a9b38 service glance] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:23.853968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a82f94-c79a-455d-a4b6-efcc144a9b38 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:23.864352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a82f94-c79a-455d-a4b6-efcc144a9b38 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:23.865927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a82f94-c79a-455d-a4b6-efcc144a9b38 service glance] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:23.877208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1160/2309] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:23.950316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be3052-5f6e-41bd-a3da-a7954be1e42a None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:23.950598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be3052-5f6e-41bd-a3da-a7954be1e42a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:23.950898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be3052-5f6e-41bd-a3da-a7954be1e42a None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:23.952242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1150/2310] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:09:23.957019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e4e7-14f9-4ebc-a3dd-02831df5b474 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:23.957306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e4e7-14f9-4ebc-a3dd-02831df5b474 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:23.957581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e4e7-14f9-4ebc-a3dd-02831df5b474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:23.978705 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d1e3e4e7-14f9-4ebc-a3dd-02831df5b474 None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:23.979903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d1e3e4e7-14f9-4ebc-a3dd-02831df5b474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:23.986811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1161/2311] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:23.999604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be3052-5f6e-41bd-a3da-a7954be1e42a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:24.001324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-01be3052-5f6e-41bd-a3da-a7954be1e42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:24.014142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be3052-5f6e-41bd-a3da-a7954be1e42a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:24.017652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8971c-096d-4b63-a8ac-d9e383b2a4c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:24.018238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8971c-096d-4b63-a8ac-d9e383b2a4c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:24.018512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8971c-096d-4b63-a8ac-d9e383b2a4c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:24.018782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8971c-096d-4b63-a8ac-d9e383b2a4c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:24.026520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8971c-096d-4b63-a8ac-d9e383b2a4c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:24.027978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8971c-096d-4b63-a8ac-d9e383b2a4c6 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:24.035579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1151/2312] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:24.117063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0717871-c17b-49cd-af98-6f532ba020f3 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:24.117339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0717871-c17b-49cd-af98-6f532ba020f3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:24.117610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0717871-c17b-49cd-af98-6f532ba020f3 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:24.118955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1162/2313] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:09:24.123814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85ce2-fe09-4194-8a03-220b484a74f4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:24.124101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85ce2-fe09-4194-8a03-220b484a74f4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:24.124370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85ce2-fe09-4194-8a03-220b484a74f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:24.144102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0ce85ce2-fe09-4194-8a03-220b484a74f4 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:24.145343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0ce85ce2-fe09-4194-8a03-220b484a74f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:24.155196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1152/2314] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:09:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:24.166299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0717871-c17b-49cd-af98-6f532ba020f3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:24.177045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0717871-c17b-49cd-af98-6f532ba020f3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:24.180760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:24.181347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:24.181623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:24.181894 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:24.191660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:24.193106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:24.203900 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1163/2315] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:26.714829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:26.715313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:26.715431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:26.734581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:26.735798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:26.743727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1153/2316] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:26.755661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:26.767037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076e741-f157-41fc-99d6-ad96a5c30446 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:26.771998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:26.772794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:26.773223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:26.773499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:26.781271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:26.782879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:26.791656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1164/2317] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:09:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:09:28.189724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:28.203621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dd176-faa9-441a-adc8-04dd1e0e72c1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:28.210207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:28.210913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:28.211254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:28.211599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:28.213457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:28.277614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:28.279218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f3027-617e-4107-adc8-d9cd60b8e700 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:28.289442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1154/2318] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:36.787634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=41 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:38.284202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=81 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:43.793626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:43.806192 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066a4c0-cd8a-4364-be77-eb3e4874f48d service swift] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:43.811553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:43.812134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:43.812406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:43.812688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:43.814393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:43.870790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:43.872357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aac859-258e-48a3-824e-0b20192ee7ba service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:43.881467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1165/2319] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:09:48.740279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:48.740779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:48.740919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:48.760659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] MFA Rules not processed for user `a3dac44116784a9e963a69e4793bac4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:48.762015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:48.768766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1155/2320] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:53.877349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=28 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:55.606059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:55.606666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:55.606666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:55.626911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:09:55.628190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:55.636625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1166/2321] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 17 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:09:55.648813 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:55.661549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9227861-a1fd-41cf-bf8c-e7603f4e9a80 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:55.666849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:55.667784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:55.668211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:55.668616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:55.677621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:55.679516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:55.687973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1156/2322] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:09:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:09:57.160615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:09:57.170547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e3bd8-d78e-4cd6-8bd5-0c1de571e1c0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:09:57.174184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:09:57.174781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:09:57.175059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:09:57.175332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:09:57.177314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:09:57.187858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:09:57.189325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:09:57.198658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1167/2323] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:04.804494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:04.805950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:04.824267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b4698-7551-491e-8cb0-4afcb38e1c35 service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:04.829704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:04.830281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:04.830554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:04.830821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:04.832474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:04.843078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:04.844641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262c219-a62c-446d-907b-8d4c99ea0d93 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:04.853398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1157/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:07.194576 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=9 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:07.956802 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:07.970436 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acc0df-c7ac-422b-b1dd-29bff9c83468 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:07.975942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:07.976547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:07.976869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:07.977133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:07.979166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:07.991908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:07.994078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:08.016399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1168/2325] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:14.849320 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=15 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:18.000480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:18.562237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:18.562733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:18.562831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:18.581572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] MFA Rules not processed for user `1e3b1e0f4888400b89c47b3e44443228`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:10:18.582748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:18.590084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1158/2326] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:10:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:10:18.898496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:18.907505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e09196-1358-41e3-b961-57af279ef038 service nova] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:18.911097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:18.911684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:18.911961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:18.912234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:18.913932 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:18.923825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:18.925283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:18.933278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1169/2327] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:19.245660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:19.247056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:19.315967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21524d45-6ede-4cdb-bea2-a6c9d2b705ae None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:19.321795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:19.322486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:19.322762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:19.323032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:19.324764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:19.335573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:19.337375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feaa152-1c4a-474f-b730-3ed8c5cd35c0 service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:19.346871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1159/2328] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:26.744035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:10:26.757344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f7941-a78e-4789-8b18-19a5c3f7cc96 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:10:26.763552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:10:26.764154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:10:26.764439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:10:26.764743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:10:26.766496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:10:26.778813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:10:26.780401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:10:26.790977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1170/2329] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:10:29.342880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=35 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:36.786427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=12 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:01.325182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:01.325717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:01.325717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:01.345261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] MFA Rules not processed for user `1dc5f1d34cad437ea15efb98a0a1ca31`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:01.346489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:01.400790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1160/2330] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:06.343681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:06.361749 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e18bdb-97f0-40a9-af72-b5fd26e016e0 service nova] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:06.368372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:06.368924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:06.369406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:06.369692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:06.371514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:06.392296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:06.394876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3583639-25d0-4149-9278-df7660ed5f71 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:06.407382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1171/2331] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:11.393671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=25 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:16.402321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:19.504854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:19.516528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf131e-aa35-48b1-a3b8-a7ee80867b33 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:19.521090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:19.521988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:19.522458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:19.522729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:19.526256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:19.592528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:19.594123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:19.604478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1161/2332] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:22.004269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.004835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.004835 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.027024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] MFA Rules not processed for user `4b5e98fb43a143ea8cfa30990b4fffc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:22.028588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:22.044264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1172/2333] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:22.682314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:22.699740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bb46a-9607-4378-9faf-b14c980ea68e service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:22.708973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:22.711437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:22.711945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:22.712452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:22.714963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:22.727480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:22.729270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:22.738975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1162/2334] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:23.696161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.697719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.713590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c5640-d67a-4dd3-9be4-f1bd624a97f0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.718600 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.719118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.719394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.719666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.721912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.734004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.735520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.747033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1173/2335] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:11:23.777847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:23.780230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:23.801417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2a32-2f40-47a6-b354-16624636f029 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:23.806745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:23.807411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:23.808327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:23.808611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] PATH_INFO: `/v3/users/a3dac44116784a9e963a69e4793bac4b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:23.809807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3dac44116784a9e963a69e4793bac4b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:23.811754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:23.861293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3dac44116784a9e963a69e4793bac4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.870261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3dac44116784a9e963a69e4793bac4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.870929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3dac44116784a9e963a69e4793bac4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.884969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3dac44116784a9e963a69e4793bac4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.892417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-91f1282e-fc84-4eec-975b-0205a53e5feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3dac44116784a9e963a69e4793bac4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:23.899615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1163/2336] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:23 2022] DELETE /identity/v3/users/a3dac44116784a9e963a69e4793bac4b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:24.082401 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.094038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1718f0-153a-4bd9-8185-42e3e5e4b438 service placement] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.152040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.152820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.153174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.153519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] PATH_INFO: `/v3/projects/1e42842c724f480f9cdd5ea75707b43f` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.154745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e42842c724f480f9cdd5ea75707b43f)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.156766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.206765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e42842c724f480f9cdd5ea75707b43f', 'request_id': 'req-858be567-79c8-4911-a9c1-a9631f84ff5a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.213125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e42842c724f480f9cdd5ea75707b43f', 'request_id': 'req-858be567-79c8-4911-a9c1-a9631f84ff5a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.220795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e42842c724f480f9cdd5ea75707b43f', 'request_id': 'req-858be567-79c8-4911-a9c1-a9631f84ff5a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.232301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] The token cache is being invalidate because project 1e42842c724f480f9cdd5ea75707b43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:24.232593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.233302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-858be567-79c8-4911-a9c1-a9631f84ff5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:24.234787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1174/2337] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:24 2022] DELETE /identity/v3/projects/1e42842c724f480f9cdd5ea75707b43f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:24.239712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.239979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.240251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.261405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:24.262606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.315653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:24.332806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1164/2338] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:24.340753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.341044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.341343 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.362631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:24.363849 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.376776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1175/2339] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:24.385578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.397864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6a6b1-ae3c-4852-acab-7ab882e1a6a2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.403051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.403799 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.404075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.404345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.405045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:24.405339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.406943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.424562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1165/2340] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:24.426001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.437779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c5f0c-d6cd-423f-8e4c-dd9c15b60705 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.443204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.443813 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.444093 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.444432 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.445690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.447622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.481681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1176/2341] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:24.486149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.497765 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b218756-d58e-4f49-aaeb-56b037a621c2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.558164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.558764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.559037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.559307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.559923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.561854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.582327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1166/2342] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:24.587020 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.598489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b2246-753b-43a3-9d51-d0be6d2eee61 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.604449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.605079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.605354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.605623 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.606259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.607858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.621359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1177/2343] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:24.622745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.632577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387ba1-f31f-4c80-846c-dd8ee70fe9a4 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.637470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1089b0-d020-488b-a726-f37924a7a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.638064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1089b0-d020-488b-a726-f37924a7a2fd admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.638337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1089b0-d020-488b-a726-f37924a7a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.638603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1089b0-d020-488b-a726-f37924a7a2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.639262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1089b0-d020-488b-a726-f37924a7a2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.640891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1089b0-d020-488b-a726-f37924a7a2fd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.652157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1167/2344] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:24.656147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.666939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02438733-4a43-4cd2-a8d2-c1e17322ab1c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.671775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.672419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.672721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.673002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] PATH_INFO: `/v3/projects/d887a0e73c024c378a4b2b5b41a139ec/users/b9267c9f3a6442fd8bfe7552c39e356c/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.687494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d887a0e73c024c378a4b2b5b41a139ec, user_id=b9267c9f3a6442fd8bfe7552c39e356c, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.690202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.704096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1178/2345] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:24 2022] PUT /identity/v3/projects/d887a0e73c024c378a4b2b5b41a139ec/users/b9267c9f3a6442fd8bfe7552c39e356c/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:24.715177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.715480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.715784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.744319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] MFA Rules not processed for user `b9267c9f3a6442fd8bfe7552c39e356c`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:24.745504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:24.822498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1168/2346] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:24.827317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.838768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:24.903575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf3cde-b1b5-4922-b1d5-4bd61235ab7a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:24.908274 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:24.908915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:24.909196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:24.909472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:24.910099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:24.912041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:24.943141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1179/2347] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:24.948231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:24.961796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce30b10-6c52-4319-9886-9b2bec17cf8b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.012931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:25.027610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.028229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.028509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.028821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.029751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.031805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.055709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1169/2348] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:24 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:25.059646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.071578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a812ce-0c27-4c70-8855-28520e4ecfc8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.076570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.077208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.077675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.077957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.078610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.080252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.091393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1180/2349] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:25.095531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.106202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a42c59-0bca-4712-9555-0743c29740e8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.110922 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.111571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.111852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.112131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] PATH_INFO: `/v3/projects/81770fd708134064a5a1d08d988ae817/users/c23b9c5348fe457f8e0f5e48da5bce21/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.128488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81770fd708134064a5a1d08d988ae817, user_id=c23b9c5348fe457f8e0f5e48da5bce21, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.131194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.146582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1170/2350] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:25 2022] PUT /identity/v3/projects/81770fd708134064a5a1d08d988ae817/users/c23b9c5348fe457f8e0f5e48da5bce21/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:25.151293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.161224 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bcc7c-7c1e-41c3-90ca-00390ab8c7dd admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.219767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb30a5b-bbce-43fc-941f-a4a75e1a7196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.220363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30a5b-bbce-43fc-941f-a4a75e1a7196 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.220643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30a5b-bbce-43fc-941f-a4a75e1a7196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.220942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb30a5b-bbce-43fc-941f-a4a75e1a7196 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.221575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb30a5b-bbce-43fc-941f-a4a75e1a7196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.223157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb30a5b-bbce-43fc-941f-a4a75e1a7196 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.234363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1181/2351] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:25.238486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.266138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c861c6e-930d-4f5f-8723-639086dc5354 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.280561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.293575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.293864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.299198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] PATH_INFO: `/v3/projects/81770fd708134064a5a1d08d988ae817/users/c23b9c5348fe457f8e0f5e48da5bce21/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.304394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81770fd708134064a5a1d08d988ae817, user_id=c23b9c5348fe457f8e0f5e48da5bce21, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.309300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.328580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1171/2352] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:25 2022] PUT /identity/v3/projects/81770fd708134064a5a1d08d988ae817/users/c23b9c5348fe457f8e0f5e48da5bce21/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:25.340121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.340495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.345080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.417276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] MFA Rules not processed for user `c23b9c5348fe457f8e0f5e48da5bce21`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:25.418709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:25.579112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1182/2353] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:25.587797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.590229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:25.658893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d558d1-2d70-40ae-ac4d-938b5685d077 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.662492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482890b2-e0a2-4355-92fb-1f57817e66a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.663129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482890b2-e0a2-4355-92fb-1f57817e66a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.663401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482890b2-e0a2-4355-92fb-1f57817e66a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.664197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482890b2-e0a2-4355-92fb-1f57817e66a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.673430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482890b2-e0a2-4355-92fb-1f57817e66a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.674856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482890b2-e0a2-4355-92fb-1f57817e66a5 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.685649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1172/2354] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:25.841641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:25.846385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:25.906442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08d4fc-69d1-4153-bcaf-3444c4d82c37 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:25.910885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:25.911507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:25.911786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:25.912062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] PATH_INFO: `/v3/users/b9267c9f3a6442fd8bfe7552c39e356c` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:25.913216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9267c9f3a6442fd8bfe7552c39e356c)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:25.915176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:25.973429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9267c9f3a6442fd8bfe7552c39e356c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.987108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9267c9f3a6442fd8bfe7552c39e356c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.987790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9267c9f3a6442fd8bfe7552c39e356c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:25.996398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9267c9f3a6442fd8bfe7552c39e356c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.009965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9267c9f3a6442fd8bfe7552c39e356c'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:26.016704 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1183/2355] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:25 2022] DELETE /identity/v3/users/b9267c9f3a6442fd8bfe7552c39e356c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:26.021647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.022169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.022366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.050868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:26.052275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:26.121476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1173/2356] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:26.135738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:26.137180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:26.195790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9bb09-c6a1-461e-b5cf-0f3d5cb726d4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:26.199397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:26.199996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:26.200268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:26.200537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:26.211463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:26.212968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:26.223214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1184/2357] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:27.296926 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:27.298440 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:27.313680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b17f1-8e80-4196-9b54-1c1f37dbd1ad None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.321565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.322451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.322898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.323963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] PATH_INFO: `/v3/projects/d887a0e73c024c378a4b2b5b41a139ec` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.325562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d887a0e73c024c378a4b2b5b41a139ec)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.327910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.383183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:27.393625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd887a0e73c024c378a4b2b5b41a139ec', 'request_id': 'req-f9908f56-55b7-4c53-b70c-3fc5f704135d'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.402708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd887a0e73c024c378a4b2b5b41a139ec', 'request_id': 'req-f9908f56-55b7-4c53-b70c-3fc5f704135d'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.409683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd887a0e73c024c378a4b2b5b41a139ec', 'request_id': 'req-f9908f56-55b7-4c53-b70c-3fc5f704135d'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.424071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] The token cache is being invalidate because project d887a0e73c024c378a4b2b5b41a139ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:27.424364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.425662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.427879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1174/2358] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:27 2022] DELETE /identity/v3/projects/d887a0e73c024c378a4b2b5b41a139ec => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:27.433457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:27.444782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31749da7-6752-48ee-a2f1-66feeac2f51e service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:27.476721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:27.520824 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:27.521545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:27.521890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:27.522235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] PATH_INFO: `/v3/users/c23b9c5348fe457f8e0f5e48da5bce21` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:27.525151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c23b9c5348fe457f8e0f5e48da5bce21)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:27.527283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:27.580895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c23b9c5348fe457f8e0f5e48da5bce21'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.599329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c23b9c5348fe457f8e0f5e48da5bce21'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.599953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c23b9c5348fe457f8e0f5e48da5bce21'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.609899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c23b9c5348fe457f8e0f5e48da5bce21'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.623855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-12d377ad-750e-4bdb-a4aa-90f20e05f1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c23b9c5348fe457f8e0f5e48da5bce21'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:27.630654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1185/2359] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:27 2022] DELETE /identity/v3/users/c23b9c5348fe457f8e0f5e48da5bce21 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:28.049309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.050747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.064792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9908f56-55b7-4c53-b70c-3fc5f704135d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.123594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.124216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.124496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.124805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] PATH_INFO: `/v3/projects/81770fd708134064a5a1d08d988ae817` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.125913 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81770fd708134064a5a1d08d988ae817)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.127841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.174036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81770fd708134064a5a1d08d988ae817', 'request_id': 'req-178b102e-d103-40b4-ab19-504ba7997198'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:28.179584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81770fd708134064a5a1d08d988ae817', 'request_id': 'req-178b102e-d103-40b4-ab19-504ba7997198'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:28.185749 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81770fd708134064a5a1d08d988ae817', 'request_id': 'req-178b102e-d103-40b4-ab19-504ba7997198'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:28.198278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] The token cache is being invalidate because project 81770fd708134064a5a1d08d988ae817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:28.198565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:28.199188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-178b102e-d103-40b4-ab19-504ba7997198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:28.200956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1175/2360] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:28 2022] DELETE /identity/v3/projects/81770fd708134064a5a1d08d988ae817 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:28.206035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.206321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.206631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.228241 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:28.229498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.295591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1186/2361] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:28.305251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.305805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.305906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.326703 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:28.328260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.337959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1176/2362] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:28.342146 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.352943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5f0e3-2e44-4ba8-8ee0-e3e138232ade None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.357561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.358163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.358437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.358733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.359369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:28.359665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.361333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.376788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1187/2363] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:28.380933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.390824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab8ce4-094a-4f04-8a25-20f143ec74c8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.395775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.396379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.396655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.396971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.398295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.400570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.430303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1177/2364] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:28.434423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.445157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eb554-b294-471b-8dcd-dd77b2967b74 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.478842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:28.508434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.509114 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.509398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.509675 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.510317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.512299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.533582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1188/2365] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:28.537682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.548218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32f94e-be50-4871-830c-b490fdc8aa4c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.552845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.553508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.553782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.554645 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.555296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.556907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.566740 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1178/2366] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:28.570997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.580795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefc577-1de3-4bad-ac61-d6b95caf536c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.585825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36ed7e-2472-46bb-8e53-9e211c81b8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.586495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36ed7e-2472-46bb-8e53-9e211c81b8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.586772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36ed7e-2472-46bb-8e53-9e211c81b8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.587045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36ed7e-2472-46bb-8e53-9e211c81b8d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.587679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36ed7e-2472-46bb-8e53-9e211c81b8d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.589430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36ed7e-2472-46bb-8e53-9e211c81b8d7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.600193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1189/2367] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:28.604986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.614782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e4f39-ba1a-4578-a17d-5b5999b4c261 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.619981 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.620634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.620935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.621207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] PATH_INFO: `/v3/projects/fece4c44c27e4dd2992ad502af4dd6ed/users/fb28d1a233394a2e96e2f877c8aa2db9/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.635344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fece4c44c27e4dd2992ad502af4dd6ed, user_id=fb28d1a233394a2e96e2f877c8aa2db9, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.638049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.640169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:28.652894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1179/2368] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:28 2022] PUT /identity/v3/projects/fece4c44c27e4dd2992ad502af4dd6ed/users/fb28d1a233394a2e96e2f877c8aa2db9/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:11:28.662994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.663280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.664078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.693149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] MFA Rules not processed for user `fb28d1a233394a2e96e2f877c8aa2db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:28.694210 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:28.764978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1190/2369] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:28.769407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.831970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f42253-47d3-4c60-87d3-cc97853f3142 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.837272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.837868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.838141 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.838412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.839166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.841100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.869436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1180/2370] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:28.873326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.884047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76654fb8-38ac-40ae-9f18-77bace3ffeca None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.945049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.945672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.946172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.946454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.947101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.949053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:28.975015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1191/2371] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:28 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:28.977024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:28.987426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f46db-e7b0-4df9-b75d-07a143c5b10d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:28.992197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:28.992827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:28.993112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:28.993386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:28.994054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:28.995656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.006253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1181/2372] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:29.010942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.021096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fce3a2-7a67-4c75-8ba7-8301f7022562 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.026613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.027326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.027609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.027870 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] PATH_INFO: `/v3/projects/fa6e8dcdf7a94fce8e204acec55125ce/users/0ef1331f1ca94721bb44eda60db58174/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.043145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6e8dcdf7a94fce8e204acec55125ce, user_id=0ef1331f1ca94721bb44eda60db58174, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.045854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.058307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1192/2373] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:29 2022] PUT /identity/v3/projects/fa6e8dcdf7a94fce8e204acec55125ce/users/0ef1331f1ca94721bb44eda60db58174/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:29.062321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.072424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c1728f-e684-4e60-93fa-0b4598a0ad36 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.127844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9665f5-057f-4cf2-a777-a4a134936d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.128475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9665f5-057f-4cf2-a777-a4a134936d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.128768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9665f5-057f-4cf2-a777-a4a134936d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.129044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9665f5-057f-4cf2-a777-a4a134936d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.129678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9665f5-057f-4cf2-a777-a4a134936d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.131306 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9665f5-057f-4cf2-a777-a4a134936d7b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.140965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1182/2374] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:29.145058 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.152133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:29.156052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7e53f-4157-46a6-8bd3-93803ae7c633 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.161960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.162601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.162879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.163156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] PATH_INFO: `/v3/projects/fa6e8dcdf7a94fce8e204acec55125ce/users/0ef1331f1ca94721bb44eda60db58174/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.165711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6e8dcdf7a94fce8e204acec55125ce, user_id=0ef1331f1ca94721bb44eda60db58174, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.168330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.178363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1193/2375] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:29 2022] PUT /identity/v3/projects/fa6e8dcdf7a94fce8e204acec55125ce/users/0ef1331f1ca94721bb44eda60db58174/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:29.188494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.188877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.189174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.217807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] MFA Rules not processed for user `0ef1331f1ca94721bb44eda60db58174`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:29.218994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:29.267702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:29.286369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1183/2376] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:29.297645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.299413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:29.356648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf167707-9e1c-4cde-b5dc-cf49ebf9ed85 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.360261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.360877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.361153 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.361428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.370869 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.372292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.381812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1194/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:29 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:29.429916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:29.431328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:29.443677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5030ec67-6139-44f7-b60d-63a1e015443c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:29.447470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531a14f-fbb1-4018-84d2-1eca019cce9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:29.448098 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531a14f-fbb1-4018-84d2-1eca019cce9e service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:29.448396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531a14f-fbb1-4018-84d2-1eca019cce9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:29.448689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531a14f-fbb1-4018-84d2-1eca019cce9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:29.509403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531a14f-fbb1-4018-84d2-1eca019cce9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:29.511173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531a14f-fbb1-4018-84d2-1eca019cce9e service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:29.521713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1184/2378] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:32.847448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:32.910045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe08243-80b2-4446-8871-b76591846df5 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:32.915462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:32.916086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:32.916361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:32.916731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] PATH_INFO: `/v3/users/fb28d1a233394a2e96e2f877c8aa2db9` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:32.917767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb28d1a233394a2e96e2f877c8aa2db9)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:32.919676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:32.970061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb28d1a233394a2e96e2f877c8aa2db9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:32.979227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb28d1a233394a2e96e2f877c8aa2db9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:32.980054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb28d1a233394a2e96e2f877c8aa2db9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:32.987161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb28d1a233394a2e96e2f877c8aa2db9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:32.999411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb28d1a233394a2e96e2f877c8aa2db9'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.005523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1195/2379] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:32 2022] DELETE /identity/v3/users/fb28d1a233394a2e96e2f877c8aa2db9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:33.010857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.011332 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.011424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.032182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:33.033582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:33.111671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1185/2380] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:33.121884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:33.123393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:33.141677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:33.222438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be7924-9c85-4af6-a6e0-7883012e8941 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:33.227655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:33.228092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.228419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.228852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.238721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.240185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:33.251185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1196/2381] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:33.666802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:33.668269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:33.683836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020972-1bfd-4fce-b31b-9b8baeb3718b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:33.689123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:33.689742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.690017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.690292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] PATH_INFO: `/v3/projects/fece4c44c27e4dd2992ad502af4dd6ed` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.691400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fece4c44c27e4dd2992ad502af4dd6ed)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.693733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:33.739701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fece4c44c27e4dd2992ad502af4dd6ed', 'request_id': 'req-57d7accf-a3f0-4082-beb7-14557c77ce58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.745875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fece4c44c27e4dd2992ad502af4dd6ed', 'request_id': 'req-57d7accf-a3f0-4082-beb7-14557c77ce58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.752049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fece4c44c27e4dd2992ad502af4dd6ed', 'request_id': 'req-57d7accf-a3f0-4082-beb7-14557c77ce58'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.764800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] The token cache is being invalidate because project fece4c44c27e4dd2992ad502af4dd6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:33.765083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.765692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.767146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1186/2382] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:33 2022] DELETE /identity/v3/projects/fece4c44c27e4dd2992ad502af4dd6ed => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:33.771131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:33.780838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553a66c-49f3-44fd-bc5e-228d94168f2e service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:33.836139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:33.836788 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:33.837067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:33.837340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] PATH_INFO: `/v3/users/0ef1331f1ca94721bb44eda60db58174` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:33.838479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ef1331f1ca94721bb44eda60db58174)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:33.840405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:33.887425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ef1331f1ca94721bb44eda60db58174'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.894599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ef1331f1ca94721bb44eda60db58174'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.895233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ef1331f1ca94721bb44eda60db58174'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.902150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ef1331f1ca94721bb44eda60db58174'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.914288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-380c11b6-1d00-448c-bf8f-3caf5ad4abd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ef1331f1ca94721bb44eda60db58174'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:33.920127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1197/2383] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:33 2022] DELETE /identity/v3/users/0ef1331f1ca94721bb44eda60db58174 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:34.189855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.191345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:34.206986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7accf-a3f0-4082-beb7-14557c77ce58 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.216734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:34.267771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.268304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.268577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.268872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] PATH_INFO: `/v3/projects/fa6e8dcdf7a94fce8e204acec55125ce` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.269961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6e8dcdf7a94fce8e204acec55125ce)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.271862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.317231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6e8dcdf7a94fce8e204acec55125ce', 'request_id': 'req-c7684888-ab64-42a7-80f6-142175096e39'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.322649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6e8dcdf7a94fce8e204acec55125ce', 'request_id': 'req-c7684888-ab64-42a7-80f6-142175096e39'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.328713 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6e8dcdf7a94fce8e204acec55125ce', 'request_id': 'req-c7684888-ab64-42a7-80f6-142175096e39'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.341321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] The token cache is being invalidate because project fa6e8dcdf7a94fce8e204acec55125ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:34.341632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.342265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c7684888-ab64-42a7-80f6-142175096e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:34.343758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1187/2384] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:34 2022] DELETE /identity/v3/projects/fa6e8dcdf7a94fce8e204acec55125ce => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:34.348582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.348889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.349162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.368973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:34.370157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:34.400281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:34.432592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1198/2385] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:34.440369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.440651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.440962 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.461979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:34.463170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:34.471170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1188/2386] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:34.475349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.486518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e01e7-4e84-4011-833c-ae3bc0519d82 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.492532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.493176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.493461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.493733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.494388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:34.494688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.496285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.512207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1199/2387] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:34.513335 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.526341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57863562-ed0c-4bc2-976b-a664a9c0795c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.532700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.533536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.533959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.534373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.535196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.537552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.572556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1189/2388] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:11:34.573405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.583411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a614a-58f3-4ef3-931a-4d5f0b5d65e2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.641110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.641891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.642172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.642442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.643078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.645012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.668382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1200/2389] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:34.673717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.684800 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62195692-2622-4823-aaa3-64de2423667d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.693136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.693730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.694004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.694271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.695455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.697078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.707381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1190/2390] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:34.712397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.722638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392ce05-f1c7-4fb9-bd80-619759b980e6 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.727831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0aa06-588a-43d7-bf82-9e95622adb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.728580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0aa06-588a-43d7-bf82-9e95622adb97 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.728886 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0aa06-588a-43d7-bf82-9e95622adb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.729155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0aa06-588a-43d7-bf82-9e95622adb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.729810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0aa06-588a-43d7-bf82-9e95622adb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.731440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0aa06-588a-43d7-bf82-9e95622adb97 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.742639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1201/2391] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:34.746757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.756830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9d5d9-5260-4cc9-8ebe-4a108d2e6599 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.761841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.762485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.762764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.763048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] PATH_INFO: `/v3/projects/23d712ad6c964df4ac003e8804921ab2/users/185dcb3537a84b00856a037652daa750/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.778265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d712ad6c964df4ac003e8804921ab2, user_id=185dcb3537a84b00856a037652daa750, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.780950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:34.794311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1191/2392] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:34 2022] PUT /identity/v3/projects/23d712ad6c964df4ac003e8804921ab2/users/185dcb3537a84b00856a037652daa750/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:34.804567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.804892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.805172 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.832872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] MFA Rules not processed for user `185dcb3537a84b00856a037652daa750`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:34.834077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:34.896879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1202/2393] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:34.902080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:34.945689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:34.968847 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920d020-b48e-48e9-9a7e-7d9b8a38104d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:34.973577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:34.974182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:34.974462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:34.974737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:34.975363 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:34.978250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.008937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1192/2394] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:34 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:35.010431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.021371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84df752-456c-4bb6-8564-08de5879fd27 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.051248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:35.085773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.086331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.086610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.086882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.087510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.089501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.112627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1203/2395] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:35 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:35.117211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.127571 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12af86-0760-4cbb-a2c0-37e16912d54d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.132839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.133439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.133719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.134014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.134665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.136270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.147786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1193/2396] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:11:35.152029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.162162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b75f7-d1f6-439a-a599-bd6eba7b200f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.167465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.168103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.168405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.168694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] PATH_INFO: `/v3/projects/188e9c419d0540cc8c4817ef13b7ddc1/users/b06d5729c98e44fe87db9d4b66474cde/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.183943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=188e9c419d0540cc8c4817ef13b7ddc1, user_id=b06d5729c98e44fe87db9d4b66474cde, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.186651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.200532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1204/2397] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:35 2022] PUT /identity/v3/projects/188e9c419d0540cc8c4817ef13b7ddc1/users/b06d5729c98e44fe87db9d4b66474cde/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:35.204447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.214647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764cd5-a025-421e-9343-4c19b9d4e470 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.270391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6ea31-4da8-4881-8fcb-1ea6958c3c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.270983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6ea31-4da8-4881-8fcb-1ea6958c3c70 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.271257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6ea31-4da8-4881-8fcb-1ea6958c3c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.271526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6ea31-4da8-4881-8fcb-1ea6958c3c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.272156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6ea31-4da8-4881-8fcb-1ea6958c3c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.274558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6ea31-4da8-4881-8fcb-1ea6958c3c70 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.284111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1194/2398] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:35.288265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.298492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cba91-7f8a-4673-8700-f28d30270e08 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.303166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.303789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.304074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.304342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] PATH_INFO: `/v3/projects/188e9c419d0540cc8c4817ef13b7ddc1/users/b06d5729c98e44fe87db9d4b66474cde/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.306286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=188e9c419d0540cc8c4817ef13b7ddc1, user_id=b06d5729c98e44fe87db9d4b66474cde, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.308966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.319326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1205/2399] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:35 2022] PUT /identity/v3/projects/188e9c419d0540cc8c4817ef13b7ddc1/users/b06d5729c98e44fe87db9d4b66474cde/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:35.331651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.332020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.332465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.362432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] MFA Rules not processed for user `b06d5729c98e44fe87db9d4b66474cde`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:35.364027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:35.434229 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1195/2400] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:35.445589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.447388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:35.505395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7baf00-3e8a-429b-b4e8-b4bc131852b8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.509302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.509906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.510190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.510467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.519952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.521419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.531876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1206/2401] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:35.637147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:35.646449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4980f-806d-4e62-b518-36c83173e3fc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:35.650496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7244df-a3e8-4700-acb5-99aae4a34aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:35.651116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7244df-a3e8-4700-acb5-99aae4a34aff service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:35.651391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7244df-a3e8-4700-acb5-99aae4a34aff service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:35.651662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7244df-a3e8-4700-acb5-99aae4a34aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:35.687493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:35.722796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7244df-a3e8-4700-acb5-99aae4a34aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:35.724259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7244df-a3e8-4700-acb5-99aae4a34aff service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:35.735361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1196/2402] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:38.464062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.532029 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9643b7-3410-4407-a120-26a8bb1769b2 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:38.539053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:38.539690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.540365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.540365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] PATH_INFO: `/v3/users/185dcb3537a84b00856a037652daa750` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.541889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] RBAC: Authorizing `identity:delete_user(user_id=185dcb3537a84b00856a037652daa750)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:38.543827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.579384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:38.605927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185dcb3537a84b00856a037652daa750'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:38.614912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185dcb3537a84b00856a037652daa750'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:38.615655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185dcb3537a84b00856a037652daa750'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:38.625659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185dcb3537a84b00856a037652daa750'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:38.639447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185dcb3537a84b00856a037652daa750'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:38.645865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1207/2403] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:38 2022] DELETE /identity/v3/users/185dcb3537a84b00856a037652daa750 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:38.650626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.651111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.651214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.671199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:38.672435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:38.759080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1197/2404] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:38.768805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:38.770347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:38.829216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290eb82-17ca-4bf6-83d9-b288ab5be0de admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:38.832992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:38.833607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:38.833883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:38.834155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:38.844026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:38.845480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:38.856150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1208/2405] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:39.133702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:39.142123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:39.157016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6f076-a8b6-4153-bc0e-453cd81903b3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.162522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.163418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.163712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.163985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] PATH_INFO: `/v3/projects/23d712ad6c964df4ac003e8804921ab2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.165410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d712ad6c964df4ac003e8804921ab2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.167455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:39.243039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d712ad6c964df4ac003e8804921ab2', 'request_id': 'req-28165366-d262-4419-ab6f-4edac40c7e15'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.251730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d712ad6c964df4ac003e8804921ab2', 'request_id': 'req-28165366-d262-4419-ab6f-4edac40c7e15'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.259149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d712ad6c964df4ac003e8804921ab2', 'request_id': 'req-28165366-d262-4419-ab6f-4edac40c7e15'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.275726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] The token cache is being invalidate because project 23d712ad6c964df4ac003e8804921ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:39.275938 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.276656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.278504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1198/2406] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:39 2022] DELETE /identity/v3/projects/23d712ad6c964df4ac003e8804921ab2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:39.282762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:39.292100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642a7fb-59cb-4ec6-915c-3936aa8962f5 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.354962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.355700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.356193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.356707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] PATH_INFO: `/v3/users/b06d5729c98e44fe87db9d4b66474cde` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.358390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06d5729c98e44fe87db9d4b66474cde)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.360441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:39.418680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06d5729c98e44fe87db9d4b66474cde'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.426284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:39.434416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06d5729c98e44fe87db9d4b66474cde'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.435790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06d5729c98e44fe87db9d4b66474cde'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.446825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06d5729c98e44fe87db9d4b66474cde'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.463018 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f5057972-e268-49a6-9bd3-3c94ed7c658f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06d5729c98e44fe87db9d4b66474cde'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.472838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1209/2407] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:39 2022] DELETE /identity/v3/users/b06d5729c98e44fe87db9d4b66474cde => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:39.750027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:39.751556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:39.765436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165366-d262-4419-ab6f-4edac40c7e15 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:39.815468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:39.823794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:39.824427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.824988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.825321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] PATH_INFO: `/v3/projects/188e9c419d0540cc8c4817ef13b7ddc1` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.826578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=188e9c419d0540cc8c4817ef13b7ddc1)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:39.828850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:39.874075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '188e9c419d0540cc8c4817ef13b7ddc1', 'request_id': 'req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.879459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '188e9c419d0540cc8c4817ef13b7ddc1', 'request_id': 'req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.886161 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '188e9c419d0540cc8c4817ef13b7ddc1', 'request_id': 'req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.898525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] The token cache is being invalidate because project 188e9c419d0540cc8c4817ef13b7ddc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:39.898816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.899441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-423b11c5-b75f-4e1c-9d2b-869fde8d9ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:39.900934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1199/2408] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:39 2022] DELETE /identity/v3/projects/188e9c419d0540cc8c4817ef13b7ddc1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:39.905968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.906258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.906538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:39.926281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:39.927476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:39.985267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1210/2409] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:11:39.993173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:39.993535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:39.993839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.015594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:40.016912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:40.026572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1200/2410] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:40.030808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.045145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de731c-877b-4b0e-b761-c8e3e1f3a731 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.049857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.050453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.050733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.051004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.051644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:40.051944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.054159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.065865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1211/2411] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:11:40.071325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.083763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a46bda-411f-446a-857f-5c2b7e8dd1de None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.089120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.089712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.089998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.090275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.090902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.092836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.120112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1201/2412] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:40 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:40.124390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.135024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6e67c-2b53-49da-ba4f-608b47b83023 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.189503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.190096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.190374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.190642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.191255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.193533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.215149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1212/2413] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:40 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:40.219709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.231338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48e313-ec43-456c-b162-3acdb7fbce99 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.236848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.237438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.237708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.237974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.238605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.240185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.250605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1202/2414] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:40.254504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.265409 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404d397-99a1-41b1-843a-7aae6ad232a4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.270157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a93844-2bf5-4d81-a611-7f7290c73f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.270748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a93844-2bf5-4d81-a611-7f7290c73f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.271027 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a93844-2bf5-4d81-a611-7f7290c73f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.271299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a93844-2bf5-4d81-a611-7f7290c73f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.271933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a93844-2bf5-4d81-a611-7f7290c73f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.274062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a93844-2bf5-4d81-a611-7f7290c73f4b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.284532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1213/2415] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:40.289217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.298996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef6a45-7294-428d-9c6e-89d8705bd36f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.303614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.304417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.304719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.304999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] PATH_INFO: `/v3/projects/1b45ca44a4cd4402a3414351057a29a5/users/9eb45860a75c428e974df76bd94049b1/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.320441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b45ca44a4cd4402a3414351057a29a5, user_id=9eb45860a75c428e974df76bd94049b1, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.323198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.336632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1203/2416] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:40 2022] PUT /identity/v3/projects/1b45ca44a4cd4402a3414351057a29a5/users/9eb45860a75c428e974df76bd94049b1/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:40.349265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.349554 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.349832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.378721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] MFA Rules not processed for user `9eb45860a75c428e974df76bd94049b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:40.379927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:40.402147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:40.447681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1214/2417] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:40.458923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:40.461467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:40.516878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b493f-f425-4b51-b188-08f27721bb49 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:40.520442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257b21-3da6-44a3-90b5-7830c0991387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:40.521065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257b21-3da6-44a3-90b5-7830c0991387 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:40.521342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257b21-3da6-44a3-90b5-7830c0991387 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:40.521613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257b21-3da6-44a3-90b5-7830c0991387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:40.530539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257b21-3da6-44a3-90b5-7830c0991387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:40.531974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257b21-3da6-44a3-90b5-7830c0991387 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:40.540697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1204/2418] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:43.101602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.162633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf7e27-a59b-41b0-863a-a35d03fd01b9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.167732 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.168357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.168635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.168934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] PATH_INFO: `/v3/users/9eb45860a75c428e974df76bd94049b1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.170046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb45860a75c428e974df76bd94049b1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.171969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.217564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb45860a75c428e974df76bd94049b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.224299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb45860a75c428e974df76bd94049b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.224923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb45860a75c428e974df76bd94049b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.232587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb45860a75c428e974df76bd94049b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.245484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb45860a75c428e974df76bd94049b1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.251569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1215/2419] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:11:43 2022] DELETE /identity/v3/users/9eb45860a75c428e974df76bd94049b1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:43.255954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.256422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.256552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.275825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:43.277083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.294871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:43.337796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1205/2420] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:43.353106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.354546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.401713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:43.414993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca435b-db34-4578-9884-ea994fa8bffc admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.418595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1067a-b94c-41d0-8464-2d6a4d72878d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.419216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1067a-b94c-41d0-8464-2d6a4d72878d service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.419491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1067a-b94c-41d0-8464-2d6a4d72878d service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.419763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1067a-b94c-41d0-8464-2d6a4d72878d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.428350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1067a-b94c-41d0-8464-2d6a4d72878d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.429826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1067a-b94c-41d0-8464-2d6a4d72878d service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.439019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1216/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:43.708091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.709564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.722029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff450-c529-47cc-84f7-2378e61c85b8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.727495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.728124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.728404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.728696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] PATH_INFO: `/v3/projects/1b45ca44a4cd4402a3414351057a29a5` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.729917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b45ca44a4cd4402a3414351057a29a5)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.731959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.780120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b45ca44a4cd4402a3414351057a29a5', 'request_id': 'req-80037fe8-5b7f-4694-970e-8549d367346c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.785595 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b45ca44a4cd4402a3414351057a29a5', 'request_id': 'req-80037fe8-5b7f-4694-970e-8549d367346c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.792520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b45ca44a4cd4402a3414351057a29a5', 'request_id': 'req-80037fe8-5b7f-4694-970e-8549d367346c'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.804854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] The token cache is being invalidate because project 1b45ca44a4cd4402a3414351057a29a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:11:43.805144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.805797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-80037fe8-5b7f-4694-970e-8549d367346c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:11:43.807360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1206/2422] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:11:43 2022] DELETE /identity/v3/projects/1b45ca44a4cd4402a3414351057a29a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:43.812511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.812854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.813112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.832699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:43.833927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.893529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1217/2423] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:43.902465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.902747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.903052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.922456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:43.923644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:43.931905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1207/2424] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:43.936115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.946572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeea96-e2a8-4ab7-b5f7-0b1c9619dfba None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.951151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.951742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.952016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.952269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.952944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:11:43.953249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.954830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:43.966147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1218/2425] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:11:43.970501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:43.979790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636e60c-2cbd-44e1-96bb-b237f1dd8b1f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:43.984394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:43.985011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:43.985289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:43.985559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:43.986177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:43.988090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.015471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1208/2426] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:11:43 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:44.019733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.029851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26c0d2-d037-4d6f-9323-b6cc14c57f08 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.086460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.087090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.087373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.087655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.088295 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.090276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.110305 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1219/2427] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:11:44 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:11:44.114465 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.125781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039edb9f-9617-4fef-ae34-e725d11f84d2 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.130478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.131058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.131330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.131662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.132293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.133904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.143844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1209/2428] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:44.147877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.157652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93085f0-6794-473f-8663-7ea23baf0586 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.162184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee287be6-13c4-44ae-84a2-09a4c2286483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.162763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee287be6-13c4-44ae-84a2-09a4c2286483 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.163609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee287be6-13c4-44ae-84a2-09a4c2286483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.163877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee287be6-13c4-44ae-84a2-09a4c2286483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.164504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee287be6-13c4-44ae-84a2-09a4c2286483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.166109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee287be6-13c4-44ae-84a2-09a4c2286483 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.176351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1220/2429] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:11:44.180596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.190509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5384-c7f9-4597-a87e-ea33967e129e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.201353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.202014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.202293 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.202568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] PATH_INFO: `/v3/projects/95d9f8d90fca4e5698859af29befc9fa/users/f83e876a03b74a2b94e3aa90570d1025/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.220688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d9f8d90fca4e5698859af29befc9fa, user_id=f83e876a03b74a2b94e3aa90570d1025, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.223378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.237656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1210/2430] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:11:44 2022] PUT /identity/v3/projects/95d9f8d90fca4e5698859af29befc9fa/users/f83e876a03b74a2b94e3aa90570d1025/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:11:44.248225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5efe53-95e9-4a5f-a7e4-6bde60242fae None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.248515 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5efe53-95e9-4a5f-a7e4-6bde60242fae None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.248823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5efe53-95e9-4a5f-a7e4-6bde60242fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.277840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ed5efe53-95e9-4a5f-a7e4-6bde60242fae None None] MFA Rules not processed for user `f83e876a03b74a2b94e3aa90570d1025`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:11:44.279065 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ed5efe53-95e9-4a5f-a7e4-6bde60242fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:44.318079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:44.343566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1221/2431] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:11:44.354952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:11:44.356717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:11:44.406419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:44.419523 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033a6e-f99e-48e9-81d0-e01e7170d473 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:11:44.424824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:11:44.425439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:11:44.425746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:11:44.426113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:11:44.435777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:11:44.437258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:11:44.446973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1211/2432] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:11:54.340612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=9 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:54.442207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:04.810834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1795f73-3b39-407d-b8c2-3430aff75e57 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:04.811349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1795f73-3b39-407d-b8c2-3430aff75e57 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:04.811447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1795f73-3b39-407d-b8c2-3430aff75e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:04.831026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-a1795f73-3b39-407d-b8c2-3430aff75e57 None None] MFA Rules not processed for user `1e3b1e0f4888400b89c47b3e44443228`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:04.832219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-a1795f73-3b39-407d-b8c2-3430aff75e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:04.886889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1222/2433] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:14.879388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=25 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:28.002824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:28.105221 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf0685-18b0-4943-9b76-da11b8734d9f service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:28.109752 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:28.110367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.110644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.110918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] PATH_INFO: `/v3/users/f83e876a03b74a2b94e3aa90570d1025` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.112013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f83e876a03b74a2b94e3aa90570d1025)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:28.113976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:28.158667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f83e876a03b74a2b94e3aa90570d1025'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.166312 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f83e876a03b74a2b94e3aa90570d1025'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.167054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f83e876a03b74a2b94e3aa90570d1025'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.179041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f83e876a03b74a2b94e3aa90570d1025'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.185711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f83e876a03b74a2b94e3aa90570d1025'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.191809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1212/2434] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:28 2022] DELETE /identity/v3/users/f83e876a03b74a2b94e3aa90570d1025 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:28.196694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.197201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.197307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.247493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:28.248608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:28.310917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1223/2435] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:28.321200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:28.322606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:28.362853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:28.396251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb4f61-ec1d-4313-b2ee-8eee04560e76 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:28.400515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c143d7-f429-4ebe-8041-0a403e9e8c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:28.401147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c143d7-f429-4ebe-8041-0a403e9e8c70 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.401421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c143d7-f429-4ebe-8041-0a403e9e8c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.401688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c143d7-f429-4ebe-8041-0a403e9e8c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.410489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c143d7-f429-4ebe-8041-0a403e9e8c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:28.411939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c143d7-f429-4ebe-8041-0a403e9e8c70 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:28.421222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1213/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:28.697731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:28.699250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:28.711158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcb346-934f-4167-ac8c-91d463c9a340 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:28.716601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:28.717264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.717540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.717814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] PATH_INFO: `/v3/projects/95d9f8d90fca4e5698859af29befc9fa` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.719375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d9f8d90fca4e5698859af29befc9fa)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:28.724089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:28.778139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d9f8d90fca4e5698859af29befc9fa', 'request_id': 'req-becd1e35-d049-4d4a-ad92-7fcbaecf9229'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.784589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d9f8d90fca4e5698859af29befc9fa', 'request_id': 'req-becd1e35-d049-4d4a-ad92-7fcbaecf9229'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.791865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d9f8d90fca4e5698859af29befc9fa', 'request_id': 'req-becd1e35-d049-4d4a-ad92-7fcbaecf9229'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.805361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] The token cache is being invalidate because project 95d9f8d90fca4e5698859af29befc9fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:28.805659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.806297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-becd1e35-d049-4d4a-ad92-7fcbaecf9229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:28.807785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1224/2437] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:28 2022] DELETE /identity/v3/projects/95d9f8d90fca4e5698859af29befc9fa => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:28.812648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.812968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.813244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:28.862225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:28.863410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:28.921642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1214/2438] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:28.985402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:28.985692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:28.985974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.007619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:29.009517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:29.017509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1225/2439] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:29.021657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.031387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e97bf3-9d80-4075-827a-c3891c816aa8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.036049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.036701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.036984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.037258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.037923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:29.038223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.039961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.051554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1215/2440] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:29.056571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.065559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7c9b-bc0c-4169-ac5e-eec83e713dbc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.070387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.071549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.071821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.072083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.072728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.074596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.100800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1226/2441] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:29 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:29.105623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.115989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad07946-59e2-4bf9-ad49-1c2b7127edc6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.172730 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.173324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.173596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.173864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.174532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.176526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.198300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1216/2442] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:29 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:29.202344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.212538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cabe0-b9d3-4035-8615-29778855fb2b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.217140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.217729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.218007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.218281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.218960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.220531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.231308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1227/2443] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:29.235808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.246433 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344d458-0120-486e-8acb-9efa53ec3036 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.250972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68be8f-fc37-4761-8ef8-6c8f3855a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.251557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68be8f-fc37-4761-8ef8-6c8f3855a25e admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.251831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68be8f-fc37-4761-8ef8-6c8f3855a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.252100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68be8f-fc37-4761-8ef8-6c8f3855a25e admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.252790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a68be8f-fc37-4761-8ef8-6c8f3855a25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.254378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a68be8f-fc37-4761-8ef8-6c8f3855a25e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.264759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1217/2444] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:29.268665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.278405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf47b9c-e500-4bbe-9f32-17f18d569773 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.282963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.283540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.283809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.284125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] PATH_INFO: `/v3/projects/3c9626a914f54f7798154c32bf4ebf31/users/09be036a734e460eb2b839a0ad7df1f8/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.299145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9626a914f54f7798154c32bf4ebf31, user_id=09be036a734e460eb2b839a0ad7df1f8, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.301823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.303941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:29.316212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1228/2445] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:29 2022] PUT /identity/v3/projects/3c9626a914f54f7798154c32bf4ebf31/users/09be036a734e460eb2b839a0ad7df1f8/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:29.326965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.327250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.327525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.340473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:29.356039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] MFA Rules not processed for user `09be036a734e460eb2b839a0ad7df1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:29.357279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:29.425509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1218/2446] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:29.436862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:29.438553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:29.493723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3b9e-3317-458c-a0a5-4a4f65709d15 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:29.497384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d1012-8bbd-4020-b61b-2f04c50b6cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:29.497977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d1012-8bbd-4020-b61b-2f04c50b6cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:29.498255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d1012-8bbd-4020-b61b-2f04c50b6cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:29.498528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d1012-8bbd-4020-b61b-2f04c50b6cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:29.506538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d1012-8bbd-4020-b61b-2f04c50b6cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:29.507993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d1012-8bbd-4020-b61b-2f04c50b6cd3 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:29.516599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1229/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:39.419990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=29 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:39.512652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:51.868463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:51.929748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8397cb-1cfb-462d-8a06-4ae15550ced3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:51.934349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:51.934967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:51.935239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:51.935507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] PATH_INFO: `/v3/users/09be036a734e460eb2b839a0ad7df1f8` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:51.936593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09be036a734e460eb2b839a0ad7df1f8)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:51.938604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:51.983299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09be036a734e460eb2b839a0ad7df1f8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.990874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09be036a734e460eb2b839a0ad7df1f8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:51.992204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09be036a734e460eb2b839a0ad7df1f8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.004191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09be036a734e460eb2b839a0ad7df1f8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.011574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09be036a734e460eb2b839a0ad7df1f8'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.017330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1219/2448] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:12:51 2022] DELETE /identity/v3/users/09be036a734e460eb2b839a0ad7df1f8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:52.021709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.022180 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.022294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.041100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:52.042286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.099882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1230/2449] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:52.109659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.111048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.187734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330348db-1dd1-422f-b9a0-cb7ef5d6276b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.191619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149d332-c35a-41e3-b30e-a7820bde725a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.192207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149d332-c35a-41e3-b30e-a7820bde725a service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.192483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149d332-c35a-41e3-b30e-a7820bde725a service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.192781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149d332-c35a-41e3-b30e-a7820bde725a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.202169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149d332-c35a-41e3-b30e-a7820bde725a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.203618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149d332-c35a-41e3-b30e-a7820bde725a service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.213935 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1220/2450] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:12:52.472903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.474323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.486417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fcae5-989d-4f65-84bf-a712da7eb22f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.491674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.492313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.492651 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.492889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] PATH_INFO: `/v3/projects/3c9626a914f54f7798154c32bf4ebf31` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.494101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9626a914f54f7798154c32bf4ebf31)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.496005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.543914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9626a914f54f7798154c32bf4ebf31', 'request_id': 'req-26d993b8-4b50-4dea-8a93-13be17381bac'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.550309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9626a914f54f7798154c32bf4ebf31', 'request_id': 'req-26d993b8-4b50-4dea-8a93-13be17381bac'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.557398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9626a914f54f7798154c32bf4ebf31', 'request_id': 'req-26d993b8-4b50-4dea-8a93-13be17381bac'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.568037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] The token cache is being invalidate because project 3c9626a914f54f7798154c32bf4ebf31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:12:52.568341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.568997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-26d993b8-4b50-4dea-8a93-13be17381bac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:12:52.571218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1231/2451] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:12:52 2022] DELETE /identity/v3/projects/3c9626a914f54f7798154c32bf4ebf31 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:52.577597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.577976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.578318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.587154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:52.601462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:52.602822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.659572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1221/2452] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:52.669196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.669479 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.669751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.689590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:52.690774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:52.699135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1232/2453] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:52.703308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.713504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed33d-e08d-4640-bbe7-a6cc1909b3d8 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.718182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.718776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.719055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.719330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.719971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:12:52.720273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.721927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.734012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1222/2454] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:12:52.738330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.747626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00adf4-4872-4068-b5b5-5bdabd108f30 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.752360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.753000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.753282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.753556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.754187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.756083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.783149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1233/2455] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:12:52.787215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.798000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707c106-e380-4df1-8748-f6587e5ca353 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.857657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.858286 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.858566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.858840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.859477 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.861454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.882324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1223/2456] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:52 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:52.886523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.896534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fda66-6000-4e04-8ab9-26345bb7353e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.902225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.902827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.903137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.903411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.904049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.905691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.916264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1234/2457] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:52.920069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.929965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb7fa-38fd-4a06-98e3-56571fcf6e4d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.934519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae355e10-e075-466e-a57d-ea1d0599e335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.935096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355e10-e075-466e-a57d-ea1d0599e335 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.935369 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355e10-e075-466e-a57d-ea1d0599e335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.935639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae355e10-e075-466e-a57d-ea1d0599e335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.936262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae355e10-e075-466e-a57d-ea1d0599e335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.937881 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae355e10-e075-466e-a57d-ea1d0599e335 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.948532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1224/2458] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:52.952352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:52.962476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dd79b-44e3-44f8-97b0-765957e45a81 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:52.966926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:52.967553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:52.967826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:52.968096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] PATH_INFO: `/v3/projects/f58304cb0e1a42ce9bbc5ea8a3e58899/users/91f2a01f833d4751a6f81d6a8f11b8ed/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:52.982774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f58304cb0e1a42ce9bbc5ea8a3e58899, user_id=91f2a01f833d4751a6f81d6a8f11b8ed, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:52.985455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:52.998171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1235/2459] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:52 2022] PUT /identity/v3/projects/f58304cb0e1a42ce9bbc5ea8a3e58899/users/91f2a01f833d4751a6f81d6a8f11b8ed/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:53.003282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:53.003569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:53.003844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:53.022507 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:53.024415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:53.082957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1225/2460] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:53.093386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:53.095112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:53.117420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:53.155094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bc2d4-69aa-40da-9cbc-2d8147f1fedb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:53.158958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4213f19-76cd-4490-8b82-90322b0a6f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:53.159483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4213f19-76cd-4490-8b82-90322b0a6f07 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:53.159804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4213f19-76cd-4490-8b82-90322b0a6f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:53.160059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4213f19-76cd-4490-8b82-90322b0a6f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:53.172158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4213f19-76cd-4490-8b82-90322b0a6f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:53.173567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4213f19-76cd-4490-8b82-90322b0a6f07 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:53.183135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1236/2461] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:12:53.420310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:53.422046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:53.436149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:53.545772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca339fb3-32ba-4299-9897-04bdf4e7229f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:53.551311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:53.551912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:53.552191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:53.552462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:53.554241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:53.617918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:53.619524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:53.629397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1226/2462] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:12:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:12:57.541183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c501052-2680-4102-9100-2113efcc171c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.541680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c501052-2680-4102-9100-2113efcc171c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.541961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c501052-2680-4102-9100-2113efcc171c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.579166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0c501052-2680-4102-9100-2113efcc171c None None] MFA Rules not processed for user `91f2a01f833d4751a6f81d6a8f11b8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:12:57.580817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0c501052-2680-4102-9100-2113efcc171c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:12:57.663040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1237/2463] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:12:57.668167 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.683283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af613f85-3adc-4090-b290-342a8d2b67ce service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.687777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.688373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.688652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.688955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.689611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.691804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.720611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1227/2464] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:57.724831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c501052-2680-4102-9100-2113efcc171c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.736634 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c501052-2680-4102-9100-2113efcc171c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.819317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.820391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.820658 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.821026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.821707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.823821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.847627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1238/2465] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:12:57 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:12:57.852305 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.869887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a04e6-efaa-4d07-87cc-6ce07e6e8e96 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.875275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.876161 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.876437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.876743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.877542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.879169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.896552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1228/2466] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:57.901000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.911543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223e236-5b10-4691-b9a6-dcf632c1af88 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:57.916959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:57.917619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:57.917922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:57.918195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] PATH_INFO: `/v3/projects/3047ba43ac51409aa77e9608816f677d/users/077b25d15fcc42e2b45a44c5f4665e4b/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:57.934945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3047ba43ac51409aa77e9608816f677d, user_id=077b25d15fcc42e2b45a44c5f4665e4b, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:57.937591 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:57.952214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1239/2467] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:57 2022] PUT /identity/v3/projects/3047ba43ac51409aa77e9608816f677d/users/077b25d15fcc42e2b45a44c5f4665e4b/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:12:57.956180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:57.967397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbb669-18e7-4458-aa17-ea2f769d9b24 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:58.017532 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:58.040038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ccb0ab-9b3c-4078-bd66-af7b1952edb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:58.040729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccb0ab-9b3c-4078-bd66-af7b1952edb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.041077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccb0ab-9b3c-4078-bd66-af7b1952edb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.041487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ccb0ab-9b3c-4078-bd66-af7b1952edb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.042222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccb0ab-9b3c-4078-bd66-af7b1952edb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.043869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ccb0ab-9b3c-4078-bd66-af7b1952edb8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.055324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1229/2468] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:12:58.059616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:12:58.071468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f1f69-f036-4bb2-9d7d-85a519f9ea26 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:12:58.076480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:12:58.077182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:12:58.077466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:12:58.077779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] PATH_INFO: `/v3/projects/3047ba43ac51409aa77e9608816f677d/users/077b25d15fcc42e2b45a44c5f4665e4b/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:12:58.081356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3047ba43ac51409aa77e9608816f677d, user_id=077b25d15fcc42e2b45a44c5f4665e4b, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:12:58.086314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:12:58.098744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1240/2469] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:12:58 2022] PUT /identity/v3/projects/3047ba43ac51409aa77e9608816f677d/users/077b25d15fcc42e2b45a44c5f4665e4b/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:13:02.230035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae7c3-f20d-4211-84d5-56352ced8bf8 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.230512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae7c3-f20d-4211-84d5-56352ced8bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.230627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ae7c3-f20d-4211-84d5-56352ced8bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.259773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b59ae7c3-f20d-4211-84d5-56352ced8bf8 None None] MFA Rules not processed for user `077b25d15fcc42e2b45a44c5f4665e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:02.260927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b59ae7c3-f20d-4211-84d5-56352ced8bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.331715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1230/2470] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:02.345871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:02.347616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:02.354806 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:02.425578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c53d6-7121-4c5e-b0c5-41faf50d2b3a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:02.431101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:02.431703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:02.432005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:02.432276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:02.490681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:02.493195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:02.502817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1241/2471] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:12.328272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=42 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:12.513391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=54 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:15.621528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.622020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.622113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.656094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:15.657402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:15.719436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1231/2472] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:13:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:15.768327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:15.769877 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:15.839074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb9938-5cce-4571-b8d1-5c24cc1971bb service nova] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:15.843825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28fb3b4-dd01-4484-8aa2-243e1f9bdd61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:15.844421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fb3b4-dd01-4484-8aa2-243e1f9bdd61 service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:15.844720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fb3b4-dd01-4484-8aa2-243e1f9bdd61 service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:15.844991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28fb3b4-dd01-4484-8aa2-243e1f9bdd61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:15.853124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fb3b4-dd01-4484-8aa2-243e1f9bdd61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:15.854978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28fb3b4-dd01-4484-8aa2-243e1f9bdd61 service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:15.862842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1242/2473] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:13:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:13:25.098210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:25.099724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.196064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4685ee9-fb76-45ec-a9f3-ef1613945350 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:25.202238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:25.202830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:25.203123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:25.203401 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:25.206040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:25.233874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:25.235771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2039b4-c36e-45aa-8383-6ac7fcab6fb7 service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:25.253040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1232/2474] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:25.859776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=32 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:26.358957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594362f-8bd1-45b1-8034-f2da86d93464 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:26.359284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594362f-8bd1-45b1-8034-f2da86d93464 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:26.359516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594362f-8bd1-45b1-8034-f2da86d93464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:26.379783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-2594362f-8bd1-45b1-8034-f2da86d93464 None None] MFA Rules not processed for user `91f2a01f833d4751a6f81d6a8f11b8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:26.381039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2594362f-8bd1-45b1-8034-f2da86d93464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:26.400907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1243/2475] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:13:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:34.884542 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b25ad9-7872-4a45-9357-2be9679f3e73 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:34.885146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b25ad9-7872-4a45-9357-2be9679f3e73 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:34.885263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b25ad9-7872-4a45-9357-2be9679f3e73 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:34.886686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1233/2476] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:13:34.893778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214ff8-7791-45af-8fd5-769c3bdaf248 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:34.894239 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214ff8-7791-45af-8fd5-769c3bdaf248 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:34.894492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df214ff8-7791-45af-8fd5-769c3bdaf248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:34.914307 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-df214ff8-7791-45af-8fd5-769c3bdaf248 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:34.915521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-df214ff8-7791-45af-8fd5-769c3bdaf248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:34.925364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1244/2477] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:13:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:34.937541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b25ad9-7872-4a45-9357-2be9679f3e73 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:34.939237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-48b25ad9-7872-4a45-9357-2be9679f3e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:34.998054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b25ad9-7872-4a45-9357-2be9679f3e73 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.001653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f8996-9a52-4d08-a700-fa21a35faa31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.002250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8996-9a52-4d08-a700-fa21a35faa31 service glance] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.002524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8996-9a52-4d08-a700-fa21a35faa31 service glance] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.002795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8996-9a52-4d08-a700-fa21a35faa31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.016513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f8996-9a52-4d08-a700-fa21a35faa31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.017984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f8996-9a52-4d08-a700-fa21a35faa31 service glance] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.029073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1234/2478] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:35.098858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4f8a2-269f-4f72-abb0-0a3a4a0d590c None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.098858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4f8a2-269f-4f72-abb0-0a3a4a0d590c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.099130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4f8a2-269f-4f72-abb0-0a3a4a0d590c None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.100550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1245/2479] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:13:35.105548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f05ecc-b7d3-4aee-97d4-95e33d9ac9b3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.105837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f05ecc-b7d3-4aee-97d4-95e33d9ac9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.106120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f05ecc-b7d3-4aee-97d4-95e33d9ac9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.127075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-21f05ecc-b7d3-4aee-97d4-95e33d9ac9b3 None None] MFA Rules not processed for user `ec498f2822ba4dc9b05a0866e885b5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:35.128262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21f05ecc-b7d3-4aee-97d4-95e33d9ac9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:35.132400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:35.180871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1235/2480] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:35.193592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4f8a2-269f-4f72-abb0-0a3a4a0d590c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.195291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2fb4f8a2-269f-4f72-abb0-0a3a4a0d590c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:35.209026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4f8a2-269f-4f72-abb0-0a3a4a0d590c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.212609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99194a1-50fa-4d7e-97a2-5a18fc9d7302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.213229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99194a1-50fa-4d7e-97a2-5a18fc9d7302 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.213539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99194a1-50fa-4d7e-97a2-5a18fc9d7302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.213834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99194a1-50fa-4d7e-97a2-5a18fc9d7302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.221032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99194a1-50fa-4d7e-97a2-5a18fc9d7302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.222448 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99194a1-50fa-4d7e-97a2-5a18fc9d7302 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.229997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1246/2481] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:35.316929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e4fb-5e27-4975-83c4-287e7ba28c11 None None] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.317211 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e4fb-5e27-4975-83c4-287e7ba28c11 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.317482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364e4fb-5e27-4975-83c4-287e7ba28c11 None None] PATH_INFO: `/` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.318834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1236/2482] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:13:35.324059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac49b-3dc4-4539-9973-5f81ce782ec0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.324338 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac49b-3dc4-4539-9973-5f81ce782ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.324605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac49b-3dc4-4539-9973-5f81ce782ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.344242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c85ac49b-3dc4-4539-9973-5f81ce782ec0 None None] MFA Rules not processed for user `ec86be5855254c8abc10fb636add2749`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:35.345458 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c85ac49b-3dc4-4539-9973-5f81ce782ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:35.354539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1247/2483] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:35.365398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e4fb-5e27-4975-83c4-287e7ba28c11 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:35.377160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364e4fb-5e27-4975-83c4-287e7ba28c11 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:35.380763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:35.381354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:35.381628 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:35.381896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:35.392142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:35.393601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:35.405004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1237/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:37.133501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.133989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.134084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.153693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:13:37.155076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.162417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1248/2485] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:13:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:13:37.175005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.187034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485017c3-6348-4ad6-8752-b4cb951adbc7 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.192193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.192802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.193075 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.193344 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.201693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.203279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.211192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1238/2486] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:13:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:13:37.291559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:13:37.323138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be92e9d-dcfa-47cc-9453-82e2a57393c7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:13:37.328506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:13:37.329251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:13:37.329613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:13:37.329983 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:13:37.342299 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:13:37.427032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:13:37.428582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:13:37.441411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1249/2487] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:13:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:13:47.207936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=31 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:47.434235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=46 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:00.071182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:00.081224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8fbc8-f29e-46fe-b147-5e640fd64824 service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:00.084762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:00.085347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:00.085618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:00.085885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:00.087561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:00.148635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:00.150031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d2818-76fd-4f3d-9d28-3de1acabe571 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:00.159550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1239/2488] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:01.843543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:01.852889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a0ac8-5ce6-469d-83d8-667714595205 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:01.857913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:01.858524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:01.858805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:01.859076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:01.869043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:01.870493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:01.880470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1250/2489] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:02.404596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.405747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.406200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.449441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:02.451209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:02.460701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1240/2490] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 17 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:02.478184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:02.491309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c2c7a-c32b-4782-a9aa-cc50086d1b72 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:02.503347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:02.504255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:02.504893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:02.505885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:02.517659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:02.519492 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:02.530119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1251/2491] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:14:07.538642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.540110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:07.556822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceeb4d-f5e3-45b9-9591-0ee26fb1b4ff None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.562866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.563501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.563777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.564097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] PATH_INFO: `/v3/users/c829f155714140bb87a78429d4e50af3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.565515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c829f155714140bb87a78429d4e50af3)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.567470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.624333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c829f155714140bb87a78429d4e50af3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.632057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c829f155714140bb87a78429d4e50af3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.632767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c829f155714140bb87a78429d4e50af3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.645529 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c829f155714140bb87a78429d4e50af3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.652494 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c829f155714140bb87a78429d4e50af3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.659057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1241/2492] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:07 2022] DELETE /identity/v3/users/c829f155714140bb87a78429d4e50af3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:07.836185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.849248 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3e35e-76fb-4f86-874a-f791ab6f3098 service swift] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:07.908599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:07.909157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:07.909439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:07.909713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] PATH_INFO: `/v3/projects/36c930f8134548e393831a0ef4034ea9` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:07.910818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c930f8134548e393831a0ef4034ea9)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:07.912769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:07.957501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c930f8134548e393831a0ef4034ea9', 'request_id': 'req-742fed2f-82ec-4159-86f6-c99c580af90e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.963579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c930f8134548e393831a0ef4034ea9', 'request_id': 'req-742fed2f-82ec-4159-86f6-c99c580af90e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.970727 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c930f8134548e393831a0ef4034ea9', 'request_id': 'req-742fed2f-82ec-4159-86f6-c99c580af90e'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.981220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] The token cache is being invalidate because project 36c930f8134548e393831a0ef4034ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:07.981452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.982127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:07.983603 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1252/2493] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:07 2022] DELETE /identity/v3/projects/36c930f8134548e393831a0ef4034ea9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:07.987763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:07.989194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:08.002101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b328e-89f0-47f4-a092-e15290079473 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.059274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.060719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.061076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.061361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] PATH_INFO: `/v3/users/91a118aa4be74fd99e7e3768e439c406` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.062533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a118aa4be74fd99e7e3768e439c406)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.064491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.066686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:08.137617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a118aa4be74fd99e7e3768e439c406'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.148842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a118aa4be74fd99e7e3768e439c406'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.149953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a118aa4be74fd99e7e3768e439c406'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.175766 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a118aa4be74fd99e7e3768e439c406'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.185581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a118aa4be74fd99e7e3768e439c406'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.194225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1242/2494] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:07 2022] DELETE /identity/v3/users/91a118aa4be74fd99e7e3768e439c406 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:08.367079 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:08.368642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:08.385397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fed2f-82ec-4159-86f6-c99c580af90e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:08.446784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:08.447414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:08.447690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:08.448011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] PATH_INFO: `/v3/projects/f0feb1aff84b469c9148c81063fcdf88` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:08.449084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0feb1aff84b469c9148c81063fcdf88)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:08.451095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:08.458103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:08.507178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0feb1aff84b469c9148c81063fcdf88', 'request_id': 'req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.515088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0feb1aff84b469c9148c81063fcdf88', 'request_id': 'req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.523089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0feb1aff84b469c9148c81063fcdf88', 'request_id': 'req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.540355 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] The token cache is being invalidate because project f0feb1aff84b469c9148c81063fcdf88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:08.540647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.541304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:08.542779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1253/2495] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:08 2022] DELETE /identity/v3/projects/f0feb1aff84b469c9148c81063fcdf88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:15.996256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:15.997742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:16.053906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54281c6-95e8-401b-9636-e03dde534b83 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:16.103889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:16.104500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:16.104815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:16.105100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:16.107462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:16.154446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:16.171316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:16.172797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:16.182458 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1243/2496] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:17.737048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:17.738301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.751203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca91c3-c9c7-48e8-aa89-23cc65fd7784 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:17.754915 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:17.755522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:17.755801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:17.756071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:17.757825 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:17.770396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:17.771834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:17.781859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1254/2497] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:19.587264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:19.589480 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:19.609929 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb987d-5697-4e12-be72-8da2e83d445b service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:19.614580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:19.615205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:19.615478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:19.615747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] PATH_INFO: `/v3/users/1e3b1e0f4888400b89c47b3e44443228` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:19.632626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e3b1e0f4888400b89c47b3e44443228)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:19.634821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:19.692021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e3b1e0f4888400b89c47b3e44443228'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.700004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e3b1e0f4888400b89c47b3e44443228'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.701350 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e3b1e0f4888400b89c47b3e44443228'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.716234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e3b1e0f4888400b89c47b3e44443228'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.726213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-997da76a-935d-4cfd-a6b8-25580bce6d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e3b1e0f4888400b89c47b3e44443228'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:19.735834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1244/2498] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:19 2022] DELETE /identity/v3/users/1e3b1e0f4888400b89c47b3e44443228 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:19.921375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:19.931853 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a071-8fb9-4470-9095-75ec5f9d2cd7 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.018296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.018903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.019185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.019467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] PATH_INFO: `/v3/projects/d9689fd1a04647cf91a73680e4b677d1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.035734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9689fd1a04647cf91a73680e4b677d1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.037771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.087453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9689fd1a04647cf91a73680e4b677d1', 'request_id': 'req-ba14534d-cde7-4b74-b406-652c08552afb'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:20.095159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9689fd1a04647cf91a73680e4b677d1', 'request_id': 'req-ba14534d-cde7-4b74-b406-652c08552afb'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:20.107707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9689fd1a04647cf91a73680e4b677d1', 'request_id': 'req-ba14534d-cde7-4b74-b406-652c08552afb'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:20.120168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] The token cache is being invalidate because project d9689fd1a04647cf91a73680e4b677d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:20.120461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:20.121131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba14534d-cde7-4b74-b406-652c08552afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:20.123217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1255/2499] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:19 2022] DELETE /identity/v3/projects/d9689fd1a04647cf91a73680e4b677d1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:20.128268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.128637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.129775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.150343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:20.151551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:20.211671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1245/2500] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:20.220276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.220644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.221136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.229967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:20.245626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:20.246888 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:20.255738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1256/2501] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:20.261048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.275500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d71fa-7e89-4abb-94b5-66a682b696c1 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.280462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.281128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.281405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.281673 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.282318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:20.282741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.284451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.297140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1246/2502] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:20.301526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.311563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49799364-a3e6-4c1b-9614-a3d6918832a0 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.316699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.317244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.317521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.317789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.318421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.320288 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.372341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1257/2503] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:20.376633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.388543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c7913-7efd-4ea8-ac1c-ed68e5bcf518 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.441019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:20.472986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.473702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.474039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.474374 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.475591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.477544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.517914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1247/2504] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:20.521816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.532390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e05761-a9ce-454a-b4c8-839c513c5b8b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.537581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.538315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.538674 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.539041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.539780 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.541536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.556470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1258/2505] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:20.561708 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.572479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375877d-f2b5-4f69-90a0-0acdc8235d7f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.578499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c215b-0f88-4a26-96c9-2210dfb6695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.579292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c215b-0f88-4a26-96c9-2210dfb6695c admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.579636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c215b-0f88-4a26-96c9-2210dfb6695c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.580001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c215b-0f88-4a26-96c9-2210dfb6695c admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.580842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c215b-0f88-4a26-96c9-2210dfb6695c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.582688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c215b-0f88-4a26-96c9-2210dfb6695c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.594498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1248/2506] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:20.598751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.609413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e88846-84fc-4d16-8643-b6bf0d9fedc2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.614221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.614880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.615169 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.615444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] PATH_INFO: `/v3/projects/da38f909cf414cbcaf6d41281cf5c971/users/3de5399c4f79468aa6b01f2cb648679a/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.636635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da38f909cf414cbcaf6d41281cf5c971, user_id=3de5399c4f79468aa6b01f2cb648679a, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.640710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.657440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1259/2507] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:20 2022] PUT /identity/v3/projects/da38f909cf414cbcaf6d41281cf5c971/users/3de5399c4f79468aa6b01f2cb648679a/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:20.668414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.668731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.669546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.702418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] MFA Rules not processed for user `3de5399c4f79468aa6b01f2cb648679a`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:20.704013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:20.777309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1249/2508] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:20.789188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.790974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:20.896580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f160657-cce8-46ad-9c3f-0664212f55b5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:20.904418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b129d00-f4cf-40be-907d-e06ace772e08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:20.905088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b129d00-f4cf-40be-907d-e06ace772e08 service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:20.905907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b129d00-f4cf-40be-907d-e06ace772e08 service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:20.905907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b129d00-f4cf-40be-907d-e06ace772e08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:20.914341 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b129d00-f4cf-40be-907d-e06ace772e08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:20.916074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b129d00-f4cf-40be-907d-e06ace772e08 service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:20.928996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1260/2509] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:20.947877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:20.949601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.057773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b017c0-3c4c-4dcb-a4fc-019e59366154 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.062617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.063143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.063903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.063903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.066041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.082427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.083889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc17df-ee58-4f5f-90f9-167b367ff11c service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.103464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1250/2510] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:21.290817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26274f56-61b5-4e46-ba28-8c5046dd0724 None None] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.291108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26274f56-61b5-4e46-ba28-8c5046dd0724 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.291388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26274f56-61b5-4e46-ba28-8c5046dd0724 None None] PATH_INFO: `/` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.293019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1261/2511] 10.222.0.16 () {54 vars in 921 bytes} [Sat Sep 17 21:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 17 21:14:21.303218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.303499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.303774 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.345359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] MFA Rules not processed for user `8df169f9be474cf6b773aaf4addf0020`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:21.346715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.352785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:21.385538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26274f56-61b5-4e46-ba28-8c5046dd0724 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.391688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-26274f56-61b5-4e46-ba28-8c5046dd0724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.446356 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1251/2512] 10.222.0.16 () {58 vars in 1018 bytes} [Sat Sep 17 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:21.465325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.467317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.480732 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badbf4f0-2a2d-45e9-b265-68cc9633d84a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.486203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.486786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.487133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.487516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.494059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26274f56-61b5-4e46-ba28-8c5046dd0724 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.497521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.499357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.499669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.500496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.500960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.501406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] PATH_INFO: `/v3/users/4b5e98fb43a143ea8cfa30990b4fffc2` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.510808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1252/2513] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:21.515451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.519766 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5e98fb43a143ea8cfa30990b4fffc2)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.522024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.525593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee818fad-97d4-4d63-a5fd-1eab62e9e8a4 service placement] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.527826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:21.532272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.533154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.533553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.535806 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.538076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:21.556449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.558237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.571258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1253/2514] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:21.581417 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5e98fb43a143ea8cfa30990b4fffc2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.589560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5e98fb43a143ea8cfa30990b4fffc2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.590338 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5e98fb43a143ea8cfa30990b4fffc2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.598161 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5e98fb43a143ea8cfa30990b4fffc2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.610920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-00e5a0ae-2c3b-44ec-892d-50554586ac8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5e98fb43a143ea8cfa30990b4fffc2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.617250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1262/2515] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:21 2022] DELETE /identity/v3/users/4b5e98fb43a143ea8cfa30990b4fffc2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:21.791010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:21.802302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bc76d-d96b-435b-a595-8623e1a3b8d6 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:21.861427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:21.861960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.862236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.862506 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] PATH_INFO: `/v3/projects/aae15e7cf5d74f9f90e067bb801c7e95` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.873142 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae15e7cf5d74f9f90e067bb801c7e95)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:21.875093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:21.918864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae15e7cf5d74f9f90e067bb801c7e95', 'request_id': 'req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.924487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae15e7cf5d74f9f90e067bb801c7e95', 'request_id': 'req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.931058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae15e7cf5d74f9f90e067bb801c7e95', 'request_id': 'req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.944297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] The token cache is being invalidate because project aae15e7cf5d74f9f90e067bb801c7e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:21.944584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.945224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-1ef9b860-bdc9-4ed8-bd0d-8151374e241b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:21.946682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1254/2516] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:21 2022] DELETE /identity/v3/projects/aae15e7cf5d74f9f90e067bb801c7e95 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:21.952166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:21.952648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:21.952769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:21.971969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:21.973181 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:22.031768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1263/2517] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:22.040819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.041108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.041384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.050520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:22.067743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:22.068978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:22.077687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1255/2518] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:22.082033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.092711 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b3338-ebda-4aa3-a074-fe4edb7e4148 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.097528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.098125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.098437 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.098696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.099345 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:22.099648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.101325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.113415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1264/2519] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:22.117690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.126854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568372da-3ae8-4fc8-8d89-914199854412 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.131442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.132070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.132348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.132617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.133276 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.135204 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.164393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1256/2520] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:22.168659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.179469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e3c32d-709b-4fc3-b97b-a39c066d9f95 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.233804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.234392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.234662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.234935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.235556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.237481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.253419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:22.258782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1265/2521] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:22.263145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.273201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561bd60-3d98-4e09-b200-89816a3e55a5 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.277743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.278379 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.278660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.279074 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.279727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.281343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.291025 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1257/2522] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:22.295844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.305702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59e034-fa41-4afe-bfce-87478eca63f0 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.311125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101af85f-c6c4-4614-a003-1b54da16a5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.311719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101af85f-c6c4-4614-a003-1b54da16a5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.312002 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101af85f-c6c4-4614-a003-1b54da16a5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.312277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101af85f-c6c4-4614-a003-1b54da16a5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.312952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101af85f-c6c4-4614-a003-1b54da16a5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.314553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101af85f-c6c4-4614-a003-1b54da16a5c8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.325490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1266/2523] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:22.329457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.339282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef2df7-e301-4ab3-9d3f-6e48d7f327d3 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.344362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.345121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.345435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.345716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] PATH_INFO: `/v3/projects/c13ecdb2540b4b2b8b007d1eed2fe9a8/users/ed5ecb421e1d4057b5f2cfa3fd3f2715/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.361899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13ecdb2540b4b2b8b007d1eed2fe9a8, user_id=ed5ecb421e1d4057b5f2cfa3fd3f2715, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.364610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.378207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1258/2524] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:22 2022] PUT /identity/v3/projects/c13ecdb2540b4b2b8b007d1eed2fe9a8/users/ed5ecb421e1d4057b5f2cfa3fd3f2715/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:22.388826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.389119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.389398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.417254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] MFA Rules not processed for user `ed5ecb421e1d4057b5f2cfa3fd3f2715`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:22.418466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:22.483159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1267/2525] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:22.487905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.549215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72305e8-85d6-4a29-9be3-084b48c684bd admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.553848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.554437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.554716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.554991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.555612 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.557590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.585533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1259/2526] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:22.589601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.600637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaf514-7df7-419a-9816-eed6fe7e9ecb None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.657301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.658358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.658358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.658781 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.659087 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.661038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.683105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1268/2527] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:14:22.688693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.699424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c126f2-540c-4024-81d3-2913706e63fa admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.705444 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.706100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.706377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.706647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.707290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.708899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.719978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1260/2528] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:22.723960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.734631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900187d9-f07c-4ffe-80b0-4c4e25f67540 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.739830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2c8d2-9012-42af-a3af-833089635fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.740435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2c8d2-9012-42af-a3af-833089635fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.740842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2c8d2-9012-42af-a3af-833089635fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.741034 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2c8d2-9012-42af-a3af-833089635fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.741708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2c8d2-9012-42af-a3af-833089635fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.743294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2c8d2-9012-42af-a3af-833089635fa9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.754149 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1269/2529] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:22.758537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.770761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af18b6-9595-4f17-b056-be987461547b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:22.776259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:22.776949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.777241 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.777508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] PATH_INFO: `/v3/projects/da6b127bdb54496b8d2ffc25106407c2/users/e3bcde8c3a8a4f05b98e61920989356f/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.795851 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6b127bdb54496b8d2ffc25106407c2, user_id=e3bcde8c3a8a4f05b98e61920989356f, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:22.799408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:22.813082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1261/2530] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:22 2022] PUT /identity/v3/projects/da6b127bdb54496b8d2ffc25106407c2/users/e3bcde8c3a8a4f05b98e61920989356f/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:22.825016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:22.826154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:22.826433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:22.863236 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] MFA Rules not processed for user `e3bcde8c3a8a4f05b98e61920989356f`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:22.864454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:22.914108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:22.936927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1270/2531] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:22.941879 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:22.996893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:23.009621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1050d-f173-4edb-9400-78d3738ca2bc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.014786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.015380 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.015659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.015931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.016554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.018493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.045216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1262/2532] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:23.049424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.060105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75102e-7892-4e52-831b-4eaea5717393 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.118082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.118682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.118958 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.119225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.119838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.121776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.142937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1271/2533] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:23 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:23.147043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.158262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded6cba-57e3-4bc8-a920-320229fa4da0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.162925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.163509 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.163783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.164056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.164711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.166308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.182303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1263/2534] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:23.186918 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.197475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383dab6a-d488-49d9-b88b-b647c2b85f74 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.202854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.203571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.203848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.204270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] PATH_INFO: `/v3/projects/bdc169d842a045c6ac95c1183e5c8fb4/users/ed2f6ab371214ff2945ee5562efa4660/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.219435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc169d842a045c6ac95c1183e5c8fb4, user_id=ed2f6ab371214ff2945ee5562efa4660, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.222178 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.235665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1272/2535] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:23 2022] PUT /identity/v3/projects/bdc169d842a045c6ac95c1183e5c8fb4/users/ed2f6ab371214ff2945ee5562efa4660/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:23.239900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.251518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9358d5-ce2a-4540-ad5b-6a8801b45589 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.306821 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe315b0-0309-47f7-b50a-32219f8aafb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.307412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe315b0-0309-47f7-b50a-32219f8aafb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.307692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe315b0-0309-47f7-b50a-32219f8aafb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.307967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe315b0-0309-47f7-b50a-32219f8aafb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.308598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe315b0-0309-47f7-b50a-32219f8aafb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.311081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe315b0-0309-47f7-b50a-32219f8aafb4 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.321014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1264/2536] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:23.324855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.335597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b6788-0eae-4cc1-86ac-5e3de7dfb26a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.340152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.340881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.341159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.341429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] PATH_INFO: `/v3/projects/bdc169d842a045c6ac95c1183e5c8fb4/users/ed2f6ab371214ff2945ee5562efa4660/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.343306 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc169d842a045c6ac95c1183e5c8fb4, user_id=ed2f6ab371214ff2945ee5562efa4660, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.345974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.356250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1273/2537] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:23 2022] PUT /identity/v3/projects/bdc169d842a045c6ac95c1183e5c8fb4/users/ed2f6ab371214ff2945ee5562efa4660/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:23.366140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.366460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.366738 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.396623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] MFA Rules not processed for user `ed2f6ab371214ff2945ee5562efa4660`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:23.397810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:23.465665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1265/2538] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:23.478258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.479935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:23.538566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68d37b-ce77-4bd9-a079-f532a69296f9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.542261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.542859 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.543137 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.543416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.553168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.554601 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.564186 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1274/2539] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:23.605528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:23.607065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:23.619466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0f7cd-4a3f-4047-8e83-11cb1667b395 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:23.624038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464efad-7750-42f1-a7cf-bb72b66d364d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:23.624659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464efad-7750-42f1-a7cf-bb72b66d364d service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:23.624961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464efad-7750-42f1-a7cf-bb72b66d364d service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:23.625236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464efad-7750-42f1-a7cf-bb72b66d364d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:23.672274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:23.689213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464efad-7750-42f1-a7cf-bb72b66d364d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:23.690933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464efad-7750-42f1-a7cf-bb72b66d364d service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:23.701215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1266/2540] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:29.746270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:29.791308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:29.810550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375756a-6150-4167-9dc7-65827ae43a48 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:29.815293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:29.815909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:29.816185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:29.816450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] PATH_INFO: `/v3/users/ed5ecb421e1d4057b5f2cfa3fd3f2715` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:29.817574 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5ecb421e1d4057b5f2cfa3fd3f2715)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:29.819473 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:29.864865 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5ecb421e1d4057b5f2cfa3fd3f2715'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:29.872905 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5ecb421e1d4057b5f2cfa3fd3f2715'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:29.873566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5ecb421e1d4057b5f2cfa3fd3f2715'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:29.880430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5ecb421e1d4057b5f2cfa3fd3f2715'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:29.892342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5ecb421e1d4057b5f2cfa3fd3f2715'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:29.898072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1275/2541] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:29 2022] DELETE /identity/v3/users/ed5ecb421e1d4057b5f2cfa3fd3f2715 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:29.902668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:29.903232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:29.903375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:29.923042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:29.924263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:29.982173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1267/2542] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:29.992125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:29.993715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:30.054480 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1c297-6468-42fc-8baf-5928c6a155e7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:30.058150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:30.058748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:30.059026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:30.059552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:30.068691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:30.070128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:30.079697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1276/2543] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:30.348760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:30.350193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:30.362176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7d66f-dea2-4ee7-9bc0-db5b42ef2c0c None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:30.366878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:30.367504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:30.367779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:30.368052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] PATH_INFO: `/v3/projects/c13ecdb2540b4b2b8b007d1eed2fe9a8` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:30.369198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13ecdb2540b4b2b8b007d1eed2fe9a8)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:30.371155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:30.421047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13ecdb2540b4b2b8b007d1eed2fe9a8', 'request_id': 'req-d52e4271-3fa5-4fd6-85fa-5f926c362295'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.426530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13ecdb2540b4b2b8b007d1eed2fe9a8', 'request_id': 'req-d52e4271-3fa5-4fd6-85fa-5f926c362295'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.432932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13ecdb2540b4b2b8b007d1eed2fe9a8', 'request_id': 'req-d52e4271-3fa5-4fd6-85fa-5f926c362295'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.445872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] The token cache is being invalidate because project c13ecdb2540b4b2b8b007d1eed2fe9a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:30.446184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.446814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.448277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1268/2544] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:30 2022] DELETE /identity/v3/projects/c13ecdb2540b4b2b8b007d1eed2fe9a8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:30.452796 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:30.462303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641283f2-3abe-457e-ab03-cd33360170ca service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:30.518096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:30.518753 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:30.519030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:30.519300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] PATH_INFO: `/v3/users/e3bcde8c3a8a4f05b98e61920989356f` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:30.520412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bcde8c3a8a4f05b98e61920989356f)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:30.522360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:30.527057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:30.582200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bcde8c3a8a4f05b98e61920989356f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.591388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bcde8c3a8a4f05b98e61920989356f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.592006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bcde8c3a8a4f05b98e61920989356f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.599589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bcde8c3a8a4f05b98e61920989356f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.611484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-d138a856-f136-4138-900f-30aaabe9aff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bcde8c3a8a4f05b98e61920989356f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.619197 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1277/2545] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:30 2022] DELETE /identity/v3/users/e3bcde8c3a8a4f05b98e61920989356f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:30.668194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:30.670267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:30.682985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4271-3fa5-4fd6-85fa-5f926c362295 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:30.740544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:30.741220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:30.741598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:30.741890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] PATH_INFO: `/v3/users/3de5399c4f79468aa6b01f2cb648679a` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:30.743324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de5399c4f79468aa6b01f2cb648679a)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:30.745577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:30.752405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:30.804846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de5399c4f79468aa6b01f2cb648679a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.813940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de5399c4f79468aa6b01f2cb648679a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.814560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de5399c4f79468aa6b01f2cb648679a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.827682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de5399c4f79468aa6b01f2cb648679a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.834697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de5399c4f79468aa6b01f2cb648679a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:30.841702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1269/2546] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:30 2022] DELETE /identity/v3/users/3de5399c4f79468aa6b01f2cb648679a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:30.846413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:30.846875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:30.846992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:30.867628 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:30.868881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:30.901060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:30.902669 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:30.935033 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1278/2547] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:30.947109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:30.949955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:30.979475 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37c054-57f5-4243-b6a0-bcab99e9f5fb admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:30.984519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:30.985177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:30.985455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:30.985733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] PATH_INFO: `/v3/projects/da6b127bdb54496b8d2ffc25106407c2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:30.986871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6b127bdb54496b8d2ffc25106407c2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:30.988835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.011845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a349a6e-a2ac-4cbf-be61-5ce2b67ba9b4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.015698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.016302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.016583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.016883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.025936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.027361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.037009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6b127bdb54496b8d2ffc25106407c2', 'request_id': 'req-0790d3ab-44ed-4d80-a889-73f6621681e0'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.037531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1279/2548] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:31.043284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6b127bdb54496b8d2ffc25106407c2', 'request_id': 'req-0790d3ab-44ed-4d80-a889-73f6621681e0'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.050551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6b127bdb54496b8d2ffc25106407c2', 'request_id': 'req-0790d3ab-44ed-4d80-a889-73f6621681e0'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.064946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] The token cache is being invalidate because project da6b127bdb54496b8d2ffc25106407c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:31.065261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.065894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.067410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1270/2549] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:30 2022] DELETE /identity/v3/projects/da6b127bdb54496b8d2ffc25106407c2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:31.071259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.082552 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5384-5074-4296-9c4f-375a1a629458 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.113785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:31.151723 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.152359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.152641 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.152997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] PATH_INFO: `/v3/users/ed2f6ab371214ff2945ee5562efa4660` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.154168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed2f6ab371214ff2945ee5562efa4660)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.156159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.207421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed2f6ab371214ff2945ee5562efa4660'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.214964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed2f6ab371214ff2945ee5562efa4660'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.215716 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed2f6ab371214ff2945ee5562efa4660'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.222891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed2f6ab371214ff2945ee5562efa4660'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.235620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1cb7c12a-a4ae-4bc8-8a6c-bc4e1334b0a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed2f6ab371214ff2945ee5562efa4660'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.242518 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1280/2550] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:31 2022] DELETE /identity/v3/users/ed2f6ab371214ff2945ee5562efa4660 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:31.324641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.326957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:31.341818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d3ab-44ed-4d80-a889-73f6621681e0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.361123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:31.419140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.419877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.420162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.420437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] PATH_INFO: `/v3/projects/da38f909cf414cbcaf6d41281cf5c971` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.421647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da38f909cf414cbcaf6d41281cf5c971)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.423611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.475345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da38f909cf414cbcaf6d41281cf5c971', 'request_id': 'req-12b6ce04-51ed-4bbb-92fa-03f845a75131'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.481524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da38f909cf414cbcaf6d41281cf5c971', 'request_id': 'req-12b6ce04-51ed-4bbb-92fa-03f845a75131'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.488697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da38f909cf414cbcaf6d41281cf5c971', 'request_id': 'req-12b6ce04-51ed-4bbb-92fa-03f845a75131'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.502852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] The token cache is being invalidate because project da38f909cf414cbcaf6d41281cf5c971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:31.503144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.503811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-12b6ce04-51ed-4bbb-92fa-03f845a75131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.505409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1271/2551] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:31 2022] DELETE /identity/v3/projects/da38f909cf414cbcaf6d41281cf5c971 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:31.510997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.511365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.511645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.534910 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:31.536100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:31.598677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1281/2552] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:31.606893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.607294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.607580 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.628858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:31.630151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:31.638080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1272/2553] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:31.642546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.655580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4c3ad-bd63-42a6-aa68-f5f3ef094d14 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.661506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.662147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.662424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.662696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.663406 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:31.672407 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.674122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.687394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1282/2554] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:31.692102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.701921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a16a0b-7689-41ba-98d9-8b718a3eb544 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.706402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.706995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.707268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.707535 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.708581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.710114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.739210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1273/2555] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:31 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:31.743419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.753728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e493e23-1cda-4ec9-9d89-4106a275388b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.785185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:31.795325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.797416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:31.821531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.822344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.822627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.822908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.823544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.826082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.852279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1283/2556] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:31 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:14:31.853359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.863333 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b1260-f71a-45c2-bfe9-ea53765d49b9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.869084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.869698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.869975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.870249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.870884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.872761 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.882573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21268d-dd69-4c51-864a-f2a72e631fc8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.884710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1284/2557] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:31.886729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.888972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.889602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.889878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.890173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] PATH_INFO: `/v3/projects/bdc169d842a045c6ac95c1183e5c8fb4` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.891299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc169d842a045c6ac95c1183e5c8fb4)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.893255 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.896847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5428f1-25db-49ad-b4bb-8dca4b7fc618 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.901416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.901992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.902266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.902532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.903751 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:31.905439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:31.915552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:31.915709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1285/2558] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:31.920105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:31.929772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575dc3c-7679-4239-8245-cd05838a5370 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:31.944135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc169d842a045c6ac95c1183e5c8fb4', 'request_id': 'req-284c5bfc-71de-4387-92cf-55cc494fdeb9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.950519 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc169d842a045c6ac95c1183e5c8fb4', 'request_id': 'req-284c5bfc-71de-4387-92cf-55cc494fdeb9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.957254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc169d842a045c6ac95c1183e5c8fb4', 'request_id': 'req-284c5bfc-71de-4387-92cf-55cc494fdeb9'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.972251 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] The token cache is being invalidate because project bdc169d842a045c6ac95c1183e5c8fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:31.972619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.973424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-284c5bfc-71de-4387-92cf-55cc494fdeb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:31.975030 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1274/2559] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:31 2022] DELETE /identity/v3/projects/bdc169d842a045c6ac95c1183e5c8fb4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:31.981403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05650c66-6d29-43ab-ae02-7d04968c8ce9 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.982008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05650c66-6d29-43ab-ae02-7d04968c8ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.982008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05650c66-6d29-43ab-ae02-7d04968c8ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:31.992736 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70fb2e-a979-4622-bfcd-0e7a7a9699bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:31.993334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70fb2e-a979-4622-bfcd-0e7a7a9699bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:31.993608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70fb2e-a979-4622-bfcd-0e7a7a9699bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:31.993875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70fb2e-a979-4622-bfcd-0e7a7a9699bc admin admin] PATH_INFO: `/v3/projects/646ac1b351ab4c58be9e80ec7bfa831c/users/4c7ccfb3172c4813a8c09b10a62d529a/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.001222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-05650c66-6d29-43ab-ae02-7d04968c8ce9 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:32.002425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-05650c66-6d29-43ab-ae02-7d04968c8ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.009708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70fb2e-a979-4622-bfcd-0e7a7a9699bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ac1b351ab4c58be9e80ec7bfa831c, user_id=4c7ccfb3172c4813a8c09b10a62d529a, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.010443 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1275/2560] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:32.013329 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70fb2e-a979-4622-bfcd-0e7a7a9699bc admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.018854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.019156 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.019398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.027243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1286/2561] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:31 2022] PUT /identity/v3/projects/646ac1b351ab4c58be9e80ec7bfa831c/users/4c7ccfb3172c4813a8c09b10a62d529a/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:32.038194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.038486 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.038768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.040432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:32.041662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.067978 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] MFA Rules not processed for user `4c7ccfb3172c4813a8c09b10a62d529a`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:32.069616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.102065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1276/2562] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:32.106248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.114822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228673f6-b4e5-4afb-b7d7-b07183e9b3ea None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.119417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.119992 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.120263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.120534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.121194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:32.121486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.123158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.134184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1287/2563] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:32.135104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1277/2564] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:14:32.139259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.139705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.140657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.149471 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d829c4e4-6265-4fcd-b0e4-21ca702b8120 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.154109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ccbdcb-5da6-4b9f-b7cd-5edbf9a62702 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.154809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.155397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.155674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.155952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.156604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.158541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.158793 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.159382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.159657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.159948 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.160575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.162475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.173397 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:32.186657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1278/2565] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:14:32.190984 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.197466 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1288/2566] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/projects => generated 401 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:32.198364 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.254916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a2313-c8c3-4f02-919b-986f3e856acd admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.259577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.260176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.260499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.260776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.261391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.262441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d9a4c-4ff6-4fcc-8f1d-d3292f395eae admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.263558 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.267367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.267956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.268235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.268509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.269176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.271102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.274964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:32.291513 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1279/2567] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:32.292577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.296204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1289/2568] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/users => generated 307 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:14:32.297244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.302658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48172ec2-a966-4450-bad5-563c931016b7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.307496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.307660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845be69-adb7-4345-97fe-3375fe41959a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.308081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.308364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.308634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.309783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.311370 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.312832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.313412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.313686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.313953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.314579 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.316411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.321503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1280/2569] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:32.326238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.327010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1290/2570] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:32.330801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.338888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7493a-c3e7-4ee3-a50b-4b868f56ff24 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.340873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe04ba-5784-48bd-990c-9ee22033da2b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.343639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.344245 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.344500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.344808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.345435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.345534 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.346190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.346472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.346748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] PATH_INFO: `/v3/projects/021ca21d18f5451bb01eeb07f50c49c7/users/9b3c704522e14514b3657bf6ab2b695f/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.347019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.357489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1281/2571] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:32.361486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.362344 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=021ca21d18f5451bb01eeb07f50c49c7, user_id=9b3c704522e14514b3657bf6ab2b695f, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.365050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.371019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be607f5-0b87-461d-8e42-0742fdc32f91 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.377189 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45508505-a85c-42d0-b7b4-0966ccf138d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.377993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45508505-a85c-42d0-b7b4-0966ccf138d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.378341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45508505-a85c-42d0-b7b4-0966ccf138d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.378758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45508505-a85c-42d0-b7b4-0966ccf138d3 admin admin] PATH_INFO: `/v3/projects/7d75e6e78d414367b1c1be9760e5d50a/users/7bd11f9291154411ac97959aa34dc423/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.380553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1291/2572] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:32 2022] PUT /identity/v3/projects/021ca21d18f5451bb01eeb07f50c49c7/users/9b3c704522e14514b3657bf6ab2b695f/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:32.384370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.395931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45508505-a85c-42d0-b7b4-0966ccf138d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d75e6e78d414367b1c1be9760e5d50a, user_id=7bd11f9291154411ac97959aa34dc423, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.398820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45508505-a85c-42d0-b7b4-0966ccf138d3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.412212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1282/2573] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:32 2022] PUT /identity/v3/projects/7d75e6e78d414367b1c1be9760e5d50a/users/7bd11f9291154411ac97959aa34dc423/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:32.424911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.425198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.425479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.451621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f5a4a9-8822-4e3f-bafb-13d3dfd65838 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.454862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] MFA Rules not processed for user `7bd11f9291154411ac97959aa34dc423`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:32.456072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.499850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:32.510553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.511096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.511379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.511657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.512907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.514528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.523839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1283/2574] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:32.525070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1292/2575] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:32.528918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.529439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.539105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d4cab-2ed5-4104-be2d-997cd6caf2ba admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.539426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a9ad-975a-428a-bb76-6640b0468da2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.543976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.544189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619a382-abb0-4f05-bf03-148988fb9a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.544715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619a382-abb0-4f05-bf03-148988fb9a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.544848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.545025 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.545125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619a382-abb0-4f05-bf03-148988fb9a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.545294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.545404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619a382-abb0-4f05-bf03-148988fb9a5f admin admin] PATH_INFO: `/v3/projects/021ca21d18f5451bb01eeb07f50c49c7/users/9b3c704522e14514b3657bf6ab2b695f/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.545953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.547379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619a382-abb0-4f05-bf03-148988fb9a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=021ca21d18f5451bb01eeb07f50c49c7, user_id=9b3c704522e14514b3657bf6ab2b695f, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.547834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.550052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619a382-abb0-4f05-bf03-148988fb9a5f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.562082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1293/2576] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:32 2022] PUT /identity/v3/projects/021ca21d18f5451bb01eeb07f50c49c7/users/9b3c704522e14514b3657bf6ab2b695f/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:32.574383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.574692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.575078 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.576583 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1284/2577] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:32.580603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.603916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] MFA Rules not processed for user `9b3c704522e14514b3657bf6ab2b695f`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:32.605156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.646250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158693c-a2ad-4bfa-b683-9c8299f9a933 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.651166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.651897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.652285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.652711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.653533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.655442 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.677051 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1285/2578] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:32.680546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1294/2579] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:32.681041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.688861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.690493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.691114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5500af7-e4ed-4083-b687-349be7307046 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.695685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.696272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.696553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.696860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.697610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.699216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.701656 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:32.712717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1286/2580] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:32.717261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.726761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e92159-f92b-439d-ab02-35fee3b70eab admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.731503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.732232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.732514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.732819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] PATH_INFO: `/v3/projects/0536abf9f1604d28a670b3d3c345f829/users/9e09b97641d246afa04aef4a7947e9b2/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.748652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0536abf9f1604d28a670b3d3c345f829, user_id=9e09b97641d246afa04aef4a7947e9b2, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.749587 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ec48f-826e-42a9-b708-6c6c42d1e1b6 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.751403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.753150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.753747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.754019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.754632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.764012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.765078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1287/2581] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:32 2022] PUT /identity/v3/projects/0536abf9f1604d28a670b3d3c345f829/users/9e09b97641d246afa04aef4a7947e9b2/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:32.765496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.769011 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.775402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1295/2582] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:32.830805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22da84-7671-45ee-9a7a-04aa7e390ad8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.835540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.836135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.836407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.836691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.837337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.838961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.853178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1288/2583] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:32.854525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.856725 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:32.868164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a431a5c-a0ea-4f5c-9444-d22474ceac9c service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.873347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55ac81-1661-4dd5-b997-60bc4137ecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.873997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55ac81-1661-4dd5-b997-60bc4137ecb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.874363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55ac81-1661-4dd5-b997-60bc4137ecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.874637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55ac81-1661-4dd5-b997-60bc4137ecb3 admin admin] PATH_INFO: `/v3/projects/0536abf9f1604d28a670b3d3c345f829/users/9e09b97641d246afa04aef4a7947e9b2/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.876580 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55ac81-1661-4dd5-b997-60bc4137ecb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0536abf9f1604d28a670b3d3c345f829, user_id=9e09b97641d246afa04aef4a7947e9b2, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:32.879282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55ac81-1661-4dd5-b997-60bc4137ecb3 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:32.890919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1296/2584] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:32 2022] PUT /identity/v3/projects/0536abf9f1604d28a670b3d3c345f829/users/9e09b97641d246afa04aef4a7947e9b2/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:32.901450 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.901748 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.902025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.915015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904fc90-9c88-4365-a68e-1b7f1a96512a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:32.930709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] MFA Rules not processed for user `9e09b97641d246afa04aef4a7947e9b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:32.931927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:32.964617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:32.965961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:32.966267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:32.966538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:32.974308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:33.005655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1297/2585] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:33.008657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:33.017174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:33.018477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:33.030745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05bbdc-d2a9-4547-aa7e-8e4b77573161 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:33.031256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:33.032739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:33.034395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c0d49-4a1b-4c01-ab09-2fcf093217fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:33.034979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c0d49-4a1b-4c01-ab09-2fcf093217fb service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:33.035254 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c0d49-4a1b-4c01-ab09-2fcf093217fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:33.035524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c0d49-4a1b-4c01-ab09-2fcf093217fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:33.041629 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1289/2586] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:32 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:33.045415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c0d49-4a1b-4c01-ab09-2fcf093217fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:33.046871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c0d49-4a1b-4c01-ab09-2fcf093217fb service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:33.056648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1298/2587] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:33.664321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:33.725008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e8f19-8dcb-4a84-a23c-82ef248d62bc service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:33.729512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:33.730122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:33.730392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:33.730660 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] PATH_INFO: `/v3/users/7bd11f9291154411ac97959aa34dc423` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:33.731951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd11f9291154411ac97959aa34dc423)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:33.733906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:33.778919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd11f9291154411ac97959aa34dc423'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:33.786237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd11f9291154411ac97959aa34dc423'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:33.786787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd11f9291154411ac97959aa34dc423'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:33.798977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd11f9291154411ac97959aa34dc423'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:33.805748 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd11f9291154411ac97959aa34dc423'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:33.812102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1290/2588] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:33 2022] DELETE /identity/v3/users/7bd11f9291154411ac97959aa34dc423 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:33.816795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:33.817084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:33.817384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:33.835936 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:33.837167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:33.898100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1299/2589] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:33.907939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:33.909376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:33.966890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e94466-70a3-4ab9-b9c0-194369b2646b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:33.970565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:33.971157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:33.971430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:33.971697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:33.981699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:33.983260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:33.988562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:33.994077 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1291/2590] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:34.260921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:34.263126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:34.277026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caef08b-bb82-492b-9c14-f2e860a01c23 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:34.282762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:34.283393 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:34.283669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:34.283940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] PATH_INFO: `/v3/projects/7d75e6e78d414367b1c1be9760e5d50a` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:34.285073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d75e6e78d414367b1c1be9760e5d50a)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:34.286968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:34.333468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d75e6e78d414367b1c1be9760e5d50a', 'request_id': 'req-7aa13254-3d64-4a76-93a6-a58754786779'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.339570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d75e6e78d414367b1c1be9760e5d50a', 'request_id': 'req-7aa13254-3d64-4a76-93a6-a58754786779'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.348555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d75e6e78d414367b1c1be9760e5d50a', 'request_id': 'req-7aa13254-3d64-4a76-93a6-a58754786779'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.359992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] The token cache is being invalidate because project 7d75e6e78d414367b1c1be9760e5d50a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:34.360285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.361140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.362544 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1300/2591] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:34 2022] DELETE /identity/v3/projects/7d75e6e78d414367b1c1be9760e5d50a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:34.366533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:34.376897 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca24a3-cca0-4cc5-bcf2-e59517f7e8fd service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:34.441387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:34.442027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:34.442302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:34.442573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] PATH_INFO: `/v3/users/9e09b97641d246afa04aef4a7947e9b2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:34.445482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e09b97641d246afa04aef4a7947e9b2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:34.447968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:34.497133 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e09b97641d246afa04aef4a7947e9b2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.504623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e09b97641d246afa04aef4a7947e9b2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.505266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e09b97641d246afa04aef4a7947e9b2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.517626 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e09b97641d246afa04aef4a7947e9b2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.524985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c0933ac3-153f-4f29-a316-c30ab465b4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e09b97641d246afa04aef4a7947e9b2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.531975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1292/2592] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:34 2022] DELETE /identity/v3/users/9e09b97641d246afa04aef4a7947e9b2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:34.819367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:34.820818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:34.834890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa13254-3d64-4a76-93a6-a58754786779 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:34.884205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:34.898428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:34.899092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:34.899369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:34.899638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] PATH_INFO: `/v3/projects/0536abf9f1604d28a670b3d3c345f829` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:34.900794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0536abf9f1604d28a670b3d3c345f829)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:34.902719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:34.947510 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0536abf9f1604d28a670b3d3c345f829', 'request_id': 'req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.953786 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0536abf9f1604d28a670b3d3c345f829', 'request_id': 'req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.960968 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0536abf9f1604d28a670b3d3c345f829', 'request_id': 'req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.971484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] The token cache is being invalidate because project 0536abf9f1604d28a670b3d3c345f829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:34.971775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.972396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9e0eedcd-a3ad-4425-9157-fd0c3b3828b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:34.973917 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1301/2593] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:34 2022] DELETE /identity/v3/projects/0536abf9f1604d28a670b3d3c345f829 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:34.979831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:34.980111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:34.980383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.000911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:35.002104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:35.058049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1293/2594] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:35.066737 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.067024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.067301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.086545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:35.087722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:35.095533 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1302/2595] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:35.099720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.109924 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca3c4-262e-4f96-b81c-10f847effd5b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.114455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.115041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.115318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.115593 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.116237 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:35.116536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.118196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.129893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1294/2596] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:35.134010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.143687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9ee77-ea33-4c01-a642-0ed9f89bde5c None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.148220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.148839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.149119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.149388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.150004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.151878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.179225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1303/2597] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:35 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:35.183409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.193706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a52df4-efd1-414f-bd54-97ebc9d22d50 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.197267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:35.254296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.254887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.255162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.255434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.256092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.258050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.278361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1295/2598] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:35 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:35.282394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.292283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d78fe-c776-4499-b15c-6d2dce4bcb1e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.296876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.297462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.297740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.298005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.298630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.300199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.310362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1304/2599] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:35.314135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.324261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f8d0c-8d2d-4557-bf03-b809bc5666eb admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.329002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92e7de-ae8c-4363-85e3-180271186a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.329586 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92e7de-ae8c-4363-85e3-180271186a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.329864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92e7de-ae8c-4363-85e3-180271186a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.330135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92e7de-ae8c-4363-85e3-180271186a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.330760 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92e7de-ae8c-4363-85e3-180271186a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.332352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92e7de-ae8c-4363-85e3-180271186a69 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.344666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1296/2600] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:35.348667 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.359005 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc05fa6-df6a-4d3c-87d6-d91f9b4e96a7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.363470 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.364100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.364374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.364647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] PATH_INFO: `/v3/projects/cba253e42b5f4444bd49a0f56df7b24f/users/f778add38e8c4bcebbc78a29651d9027/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.379187 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba253e42b5f4444bd49a0f56df7b24f, user_id=f778add38e8c4bcebbc78a29651d9027, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.381880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.395193 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1305/2601] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:35 2022] PUT /identity/v3/projects/cba253e42b5f4444bd49a0f56df7b24f/users/f778add38e8c4bcebbc78a29651d9027/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:14:35.406977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.407438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.407651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.435357 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] MFA Rules not processed for user `f778add38e8c4bcebbc78a29651d9027`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:35.436567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:35.502486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1297/2602] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:35.514449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:35.516122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:35.572145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd762c3-b69b-4401-9cd0-65419b8bb9a7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:35.576242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbbf6d-c8a1-47f5-ac9c-6333e623689f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:35.576861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbf6d-c8a1-47f5-ac9c-6333e623689f service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:35.577140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbf6d-c8a1-47f5-ac9c-6333e623689f service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:35.577410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbf6d-c8a1-47f5-ac9c-6333e623689f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:35.585503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbbf6d-c8a1-47f5-ac9c-6333e623689f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:35.586925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbbf6d-c8a1-47f5-ac9c-6333e623689f service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:35.595690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1306/2603] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:37.160419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:37.161858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:37.227400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a593aca-2554-432a-97c6-c53af266a204 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:37.231989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:37.232611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:37.232918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:37.233193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] PATH_INFO: `/v3/users/4c7ccfb3172c4813a8c09b10a62d529a` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:37.234299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7ccfb3172c4813a8c09b10a62d529a)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:37.236230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:37.254664 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:37.286375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7ccfb3172c4813a8c09b10a62d529a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.293397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7ccfb3172c4813a8c09b10a62d529a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.293993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7ccfb3172c4813a8c09b10a62d529a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.305777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7ccfb3172c4813a8c09b10a62d529a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.312499 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7ccfb3172c4813a8c09b10a62d529a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.318467 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1298/2604] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:37 2022] DELETE /identity/v3/users/4c7ccfb3172c4813a8c09b10a62d529a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:37.323071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:37.323558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:37.323672 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:37.342860 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:37.344054 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:37.352963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:37.405639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1307/2605] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:37.415403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:37.416853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:37.473829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e4150-eb26-496f-a78b-b8b38a33f0bb admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:37.477607 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:37.478131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:37.478402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:37.478673 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:37.487839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:37.489360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:37.499857 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1299/2606] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:37.758205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:37.759684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:37.772823 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2466e1-31ab-4c57-a1ed-91e678e9b44d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:37.777521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:37.778350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:37.778700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:37.779073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] PATH_INFO: `/v3/projects/646ac1b351ab4c58be9e80ec7bfa831c` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:37.780301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=646ac1b351ab4c58be9e80ec7bfa831c)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:37.782318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:37.829998 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646ac1b351ab4c58be9e80ec7bfa831c', 'request_id': 'req-f97a82a4-c15d-4086-9cf0-13ba12625a1d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.836211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646ac1b351ab4c58be9e80ec7bfa831c', 'request_id': 'req-f97a82a4-c15d-4086-9cf0-13ba12625a1d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.843733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646ac1b351ab4c58be9e80ec7bfa831c', 'request_id': 'req-f97a82a4-c15d-4086-9cf0-13ba12625a1d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.854190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] The token cache is being invalidate because project 646ac1b351ab4c58be9e80ec7bfa831c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:37.854461 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.855082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.857227 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1308/2607] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:37 2022] DELETE /identity/v3/projects/646ac1b351ab4c58be9e80ec7bfa831c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:37.861671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:37.871331 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22cec7-4e0a-4338-9070-781724d32001 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:37.925993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:37.926605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:37.926882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:37.927153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] PATH_INFO: `/v3/users/9b3c704522e14514b3657bf6ab2b695f` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:37.928242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b3c704522e14514b3657bf6ab2b695f)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:37.930175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:37.970781 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:37.978502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b3c704522e14514b3657bf6ab2b695f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.986243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b3c704522e14514b3657bf6ab2b695f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.986855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b3c704522e14514b3657bf6ab2b695f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:37.999225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b3c704522e14514b3657bf6ab2b695f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.006158 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-66d89825-8263-4c9c-b181-1a3b82e61a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b3c704522e14514b3657bf6ab2b695f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.012049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1300/2608] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:37 2022] DELETE /identity/v3/users/9b3c704522e14514b3657bf6ab2b695f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:38.280300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.281776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:38.296819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97a82a4-c15d-4086-9cf0-13ba12625a1d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.353698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.354352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.354588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.354862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] PATH_INFO: `/v3/projects/021ca21d18f5451bb01eeb07f50c49c7` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.356244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021ca21d18f5451bb01eeb07f50c49c7)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.358289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.398616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:38.405986 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021ca21d18f5451bb01eeb07f50c49c7', 'request_id': 'req-221f6f93-263e-4a01-9267-0c699e76eea4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.412760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021ca21d18f5451bb01eeb07f50c49c7', 'request_id': 'req-221f6f93-263e-4a01-9267-0c699e76eea4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.420855 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021ca21d18f5451bb01eeb07f50c49c7', 'request_id': 'req-221f6f93-263e-4a01-9267-0c699e76eea4'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.431164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] The token cache is being invalidate because project 021ca21d18f5451bb01eeb07f50c49c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:38.431453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.432086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-221f6f93-263e-4a01-9267-0c699e76eea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:38.433613 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1309/2609] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:38 2022] DELETE /identity/v3/projects/021ca21d18f5451bb01eeb07f50c49c7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:38.438511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.439036 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.439136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.458649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:38.459832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:38.517633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1301/2610] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:38.525757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.525992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.526273 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.546861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:38.548062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:38.555979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1310/2611] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:38.560164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.570047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fdc14b-5b7d-42ad-bf62-b74fce1ed68b None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.575233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.575831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.576113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.576381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.577060 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:38.577361 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.578986 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.591436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1302/2612] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:14:38.595395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.604425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc3e05-01aa-4478-a3ca-ca639af23363 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.609159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.609743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.610022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.610296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.610926 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.612847 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.640209 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1311/2613] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:38 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:38.644087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.654655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156890e-b2c5-411a-9da9-025e78a86f13 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.710479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.711071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.711345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.711611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.712239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.714172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.735671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1303/2614] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:38 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:38.739897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.751602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bf389-ef5a-47d0-8e96-ab10b22ad6ed admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.756686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.757358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.757709 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.757987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.758633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.760279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.770457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1312/2615] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:38.774484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.784484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce8853-df9e-450e-841d-b23ce564af21 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.788967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69909526-a2d9-49ae-a52c-a137adbefe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.789546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69909526-a2d9-49ae-a52c-a137adbefe46 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.789817 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69909526-a2d9-49ae-a52c-a137adbefe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.790085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69909526-a2d9-49ae-a52c-a137adbefe46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.790714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69909526-a2d9-49ae-a52c-a137adbefe46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.792289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69909526-a2d9-49ae-a52c-a137adbefe46 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.803151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1304/2616] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:38.807204 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.817528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c8470-8c64-412f-9134-2e45211b1195 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:38.822108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:38.822742 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.823022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.823296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] PATH_INFO: `/v3/projects/944e4404007045db97bd1862f24466a3/users/10e135e2662b424e98d83ab9471e9ffc/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.838147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=944e4404007045db97bd1862f24466a3, user_id=10e135e2662b424e98d83ab9471e9ffc, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:38.840868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:38.854930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1313/2617] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:38 2022] PUT /identity/v3/projects/944e4404007045db97bd1862f24466a3/users/10e135e2662b424e98d83ab9471e9ffc/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:38.860482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd3a2-87d1-47e6-97c8-288e5270967a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:38.860805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd3a2-87d1-47e6-97c8-288e5270967a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:38.861079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becd3a2-87d1-47e6-97c8-288e5270967a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:38.880549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0becd3a2-87d1-47e6-97c8-288e5270967a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:38.881771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0becd3a2-87d1-47e6-97c8-288e5270967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:38.898743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:38.942556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1305/2618] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:38.952904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:38.954589 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:39.009670 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935eb56-15d4-49eb-8dc9-6be00b35f166 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:39.013887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:39.014504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:39.014776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:39.015041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:39.023953 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:39.025438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:39.034908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1314/2619] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:14:39.881461 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:39.881993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:39.882218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:39.911682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] MFA Rules not processed for user `10e135e2662b424e98d83ab9471e9ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:39.913102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:39.978351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1306/2620] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:39.983530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:39.993226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2ccfb-1c54-444a-bab3-715c3e6a8783 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:39.997843 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:39.998435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:39.998710 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:39.998979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:39.999620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:40.002454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:40.032924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1315/2621] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:39 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:40.037007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:40.048589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce79f-cf46-42cb-bf3c-1f44b8ae2e4f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:40.105855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:40.106400 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:40.106672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:40.107479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:40.108289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:40.110225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:40.131627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1307/2622] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:40 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:40.135816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:40.146279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0778b-1f6e-4bce-89d1-0192a517306b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:40.150826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:40.151408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:40.151682 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:40.151947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:40.152583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:40.154399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:40.166121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1316/2623] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:14:40.170719 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:40.180423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594909c-1779-456e-be90-f2b2f62aac1a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:40.185650 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:40.186285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:40.186559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:40.186825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] PATH_INFO: `/v3/projects/681f4ab2823c4191b048c3eff764cf7d/users/ca58808ce10241b897f735e36491db66/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:40.201972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=681f4ab2823c4191b048c3eff764cf7d, user_id=ca58808ce10241b897f735e36491db66, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:40.204972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:40.218789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1308/2624] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:40 2022] PUT /identity/v3/projects/681f4ab2823c4191b048c3eff764cf7d/users/ca58808ce10241b897f735e36491db66/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:40.223271 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:40.233975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f9437-a78a-4485-992d-b2afb31744e1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:40.268841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:40.294183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232f50-9005-41bc-9a54-7b368d551024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:40.295285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232f50-9005-41bc-9a54-7b368d551024 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:40.295560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232f50-9005-41bc-9a54-7b368d551024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:40.295833 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232f50-9005-41bc-9a54-7b368d551024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:40.296459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef232f50-9005-41bc-9a54-7b368d551024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:40.298085 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef232f50-9005-41bc-9a54-7b368d551024 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:40.307790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1317/2625] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:40.312812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:40.323094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c339ca-0183-4450-a856-c2ee1508c1bc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:40.327667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:40.328296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:40.328570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:40.328868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] PATH_INFO: `/v3/projects/681f4ab2823c4191b048c3eff764cf7d/users/ca58808ce10241b897f735e36491db66/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:40.330731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681f4ab2823c4191b048c3eff764cf7d, user_id=ca58808ce10241b897f735e36491db66, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:40.333462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:40.343382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1309/2626] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:40 2022] PUT /identity/v3/projects/681f4ab2823c4191b048c3eff764cf7d/users/ca58808ce10241b897f735e36491db66/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:40.965358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:40.965760 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:40.966188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:40.994629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] MFA Rules not processed for user `ca58808ce10241b897f735e36491db66`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:40.995960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:41.066412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1318/2627] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:41.078348 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:41.080089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:41.086776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:41.144359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d674249-0345-45d4-86c0-bf13c0c117c8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:41.148307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:41.148874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:41.149168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:41.149425 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:41.206154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:41.207600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:41.217083 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1310/2628] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:46.810636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:46.820292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fcc7be-d7b7-41c6-bade-f310514008c4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:46.824024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858313fa-17bd-47af-9a6f-3fdf456f3d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:46.824694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858313fa-17bd-47af-9a6f-3fdf456f3d64 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:46.824990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858313fa-17bd-47af-9a6f-3fdf456f3d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:46.825356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858313fa-17bd-47af-9a6f-3fdf456f3d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:46.834692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858313fa-17bd-47af-9a6f-3fdf456f3d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:46.836143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858313fa-17bd-47af-9a6f-3fdf456f3d64 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:46.846017 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1319/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:51.212453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=46 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:56.841218 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=53 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:57.300069 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:57.301891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:57.367648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df82eb9-92e1-4f4f-a15f-62691e2d90d0 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:57.372311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:57.373040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:57.373316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:57.373587 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] PATH_INFO: `/v3/users/f778add38e8c4bcebbc78a29651d9027` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:57.375283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f778add38e8c4bcebbc78a29651d9027)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:57.377368 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:57.423113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f778add38e8c4bcebbc78a29651d9027'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.430096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f778add38e8c4bcebbc78a29651d9027'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.430686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f778add38e8c4bcebbc78a29651d9027'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.443588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f778add38e8c4bcebbc78a29651d9027'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.450631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f778add38e8c4bcebbc78a29651d9027'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:57.456663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1311/2630] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:14:57 2022] DELETE /identity/v3/users/f778add38e8c4bcebbc78a29651d9027 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:57.461055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:57.461337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:57.461610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:57.480060 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:57.481264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:57.538108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1320/2631] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:57.548274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:57.549675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:57.605700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e66aa-771e-4792-9fd5-47971d49e98f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:57.609787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbc9ec-3c0f-448a-949f-a4215d674b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:57.610376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbc9ec-3c0f-448a-949f-a4215d674b4d service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:57.610659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbc9ec-3c0f-448a-949f-a4215d674b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:57.610925 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbc9ec-3c0f-448a-949f-a4215d674b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:57.619933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbc9ec-3c0f-448a-949f-a4215d674b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:57.621397 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbc9ec-3c0f-448a-949f-a4215d674b4d service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:57.631572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1312/2632] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:14:57.919681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:57.921850 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:57.943511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6ec88-91a8-4318-aebc-0860b6b122bd None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:57.950834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:57.951842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:57.952347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:57.953176 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] PATH_INFO: `/v3/projects/cba253e42b5f4444bd49a0f56df7b24f` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:57.954965 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba253e42b5f4444bd49a0f56df7b24f)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:57.958188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.010259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba253e42b5f4444bd49a0f56df7b24f', 'request_id': 'req-9f258e1e-674a-4058-94b0-a86852666bea'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:58.017175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba253e42b5f4444bd49a0f56df7b24f', 'request_id': 'req-9f258e1e-674a-4058-94b0-a86852666bea'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:58.025199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba253e42b5f4444bd49a0f56df7b24f', 'request_id': 'req-9f258e1e-674a-4058-94b0-a86852666bea'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:58.036648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] The token cache is being invalidate because project cba253e42b5f4444bd49a0f56df7b24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:14:58.036955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:58.037571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-9f258e1e-674a-4058-94b0-a86852666bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:14:58.039045 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1321/2633] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:14:57 2022] DELETE /identity/v3/projects/cba253e42b5f4444bd49a0f56df7b24f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:58.043910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.044198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.044472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.065853 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:58.067111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:58.084933 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:58.129213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1313/2634] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:58.136403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.136703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.137112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.157549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:58.158734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:58.166493 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1322/2635] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:14:58.170623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.180314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6002e908-37be-47cd-bc9e-aafbcb1bd2cc None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.185094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.185681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.185954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.186224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.186893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:14:58.187275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.189022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.200640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1314/2636] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:14:58.205475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.214695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343067-9560-4525-bc50-cbcf884dd73d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.219991 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.220612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.220919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.221212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.221831 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.223826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.250840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1323/2637] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:58.255436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.266047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b0b08-83fa-49f5-84bd-8c11caff8a8d admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.323767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.324539 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.324642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.324915 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.325540 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.328294 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.350047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1315/2638] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:58.354158 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.364789 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7a84d-5a99-496d-aea3-d55a400cc8ca admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.370951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.371585 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.371866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.372141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.372814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.374970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.386919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1324/2639] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:58.391662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.404073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b988a-8aa1-4abc-a27e-2cad4c031a11 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.409343 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246889ac-ee43-459f-9537-91e940572cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.409934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246889ac-ee43-459f-9537-91e940572cff admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.410208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246889ac-ee43-459f-9537-91e940572cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.410474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246889ac-ee43-459f-9537-91e940572cff admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.411106 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246889ac-ee43-459f-9537-91e940572cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.412720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246889ac-ee43-459f-9537-91e940572cff admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.429292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1316/2640] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:58.433686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.445536 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45531fa-8e9a-4286-bad8-a6e29cc6ef67 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.450296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.450939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.451213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.451482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] PATH_INFO: `/v3/projects/37059c02219a4e54bdbb4e6ce3ed692e/users/5c86fb90e0554b43a6c3df9f81baff6b/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.467147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37059c02219a4e54bdbb4e6ce3ed692e, user_id=5c86fb90e0554b43a6c3df9f81baff6b, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.470231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.484868 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1325/2641] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:58 2022] PUT /identity/v3/projects/37059c02219a4e54bdbb4e6ce3ed692e/users/5c86fb90e0554b43a6c3df9f81baff6b/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:58.496858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.497194 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.497469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.531960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] MFA Rules not processed for user `5c86fb90e0554b43a6c3df9f81baff6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:58.533253 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:58.601731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1317/2642] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:58.606184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.629699 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:58.675297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52030d-4273-425d-8719-3e6de716c611 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.680031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.680883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.681216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.681500 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.682136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.684047 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.712733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1326/2643] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:58.717064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.728426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fa43b-62ec-4318-a497-d04676b439b4 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.741638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:58.791846 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.792453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.792770 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.793044 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.793692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.795613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.820786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1318/2644] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:58.824852 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.835992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc9dc4-e7c4-4f88-b1e1-2b5c3cda5558 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.842162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.843216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.843216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.843541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.844294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.845935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.857442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1327/2645] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:58.864050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.876049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79f783-ce9b-457e-b754-938be6d943a1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.881840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fc146-8410-4cc3-9fd6-d9319d8048d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.882437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc146-8410-4cc3-9fd6-d9319d8048d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.882715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc146-8410-4cc3-9fd6-d9319d8048d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.882987 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc146-8410-4cc3-9fd6-d9319d8048d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.883623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fc146-8410-4cc3-9fd6-d9319d8048d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.885338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fc146-8410-4cc3-9fd6-d9319d8048d6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.896627 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1319/2646] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:58.901244 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:58.911421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264415a-25fe-4713-8d01-31fb120e3de8 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:58.916421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:58.917125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.917402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.917680 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] PATH_INFO: `/v3/projects/bf80660456984f919172372f245db890/users/b13121c7b5bf4712820eb10457a8306b/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.935318 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf80660456984f919172372f245db890, user_id=b13121c7b5bf4712820eb10457a8306b, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:58.938945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:58.953610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1328/2647] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:58 2022] PUT /identity/v3/projects/bf80660456984f919172372f245db890/users/b13121c7b5bf4712820eb10457a8306b/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:58.963530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:58.963820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:58.964101 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:58.995446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] MFA Rules not processed for user `b13121c7b5bf4712820eb10457a8306b`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:58.996872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.086617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1320/2648] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:59.088867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.154135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56deff13-bc3d-4f4b-b16a-9810d080b725 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.159478 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.160116 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.160395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.160668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.162816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.163222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.193061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1329/2649] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:14:59 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:59.196210 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.209702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8201ef6-d539-479c-a7e1-2565fc7fb8f4 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.276733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.277358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.277640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.277917 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.278636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.281466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.307812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1321/2650] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:14:59 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:14:59.311528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.321972 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56252c-3563-4a30-8a67-d69007a4e216 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.326511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.327105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.327421 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.327693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.328328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.330043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.341035 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1330/2651] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:59.346734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.361273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6105c9-ff08-45a5-b267-f96bc640a106 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.368355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.369037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.369313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.369579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] PATH_INFO: `/v3/projects/444abe2be6ed4b79aea6e3a936d2304a/users/79339ee264be46379605742cfa085120/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.384948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:59.388744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444abe2be6ed4b79aea6e3a936d2304a, user_id=79339ee264be46379605742cfa085120, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.392064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.407446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1322/2652] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:59 2022] PUT /identity/v3/projects/444abe2be6ed4b79aea6e3a936d2304a/users/79339ee264be46379605742cfa085120/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:59.411351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.422019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8241c3f-17c3-4205-88f8-e63cbc303779 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.461901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:59.478061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7464-18b3-4561-a67a-68c552df076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.478661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7464-18b3-4561-a67a-68c552df076a admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.478964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7464-18b3-4561-a67a-68c552df076a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.479233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7464-18b3-4561-a67a-68c552df076a admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.479861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7464-18b3-4561-a67a-68c552df076a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.481477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7464-18b3-4561-a67a-68c552df076a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.491817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1331/2653] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:14:59.496130 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.506207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a608c-261c-41ed-ae9c-73e1c1d192d8 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.510727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.511352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.511625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.511904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] PATH_INFO: `/v3/projects/444abe2be6ed4b79aea6e3a936d2304a/users/79339ee264be46379605742cfa085120/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.513835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444abe2be6ed4b79aea6e3a936d2304a, user_id=79339ee264be46379605742cfa085120, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.517299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.527426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1323/2654] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:14:59 2022] PUT /identity/v3/projects/444abe2be6ed4b79aea6e3a936d2304a/users/79339ee264be46379605742cfa085120/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:14:59.538068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.538361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.538644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.569460 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] MFA Rules not processed for user `79339ee264be46379605742cfa085120`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:14:59.570669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.637911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1332/2655] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:14:59.650039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:14:59.651783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:14:59.710135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e77d9-f3fc-47f1-82a5-0ebcb044d7f4 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:14:59.713784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:14:59.714310 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:14:59.714582 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:14:59.714849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:14:59.768863 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:14:59.770290 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:14:59.779057 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1324/2656] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:01.277076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.287216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1ec15-b17d-4dd0-9acc-c85fc6759fd2 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.290808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.291405 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.291681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.291952 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.346977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.348328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.357074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1333/2657] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:01.864000 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:01.872511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863f8e2-b415-4993-ab12-a13967576f0e service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:01.876072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:01.876652 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:01.876955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:01.877223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:01.885840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:01.887270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:01.897528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1325/2658] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:03.055413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.056969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.127644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c4cd9b-7db0-4085-be01-b98c229175cd service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.132323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.133006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.133290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.133567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] PATH_INFO: `/v3/users/10e135e2662b424e98d83ab9471e9ffc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.134706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e135e2662b424e98d83ab9471e9ffc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.136626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.141775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:03.191866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e135e2662b424e98d83ab9471e9ffc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.199879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e135e2662b424e98d83ab9471e9ffc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.200519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e135e2662b424e98d83ab9471e9ffc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.207691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e135e2662b424e98d83ab9471e9ffc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.222642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e135e2662b424e98d83ab9471e9ffc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.228734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1334/2659] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/users/10e135e2662b424e98d83ab9471e9ffc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:03.418483 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.430877 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a413-dcae-4e17-8e2f-b5b1ba7853fc service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.487734 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.488352 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.488630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.488936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] PATH_INFO: `/v3/projects/944e4404007045db97bd1862f24466a3` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.490043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944e4404007045db97bd1862f24466a3)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.491941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.513787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:03.537430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944e4404007045db97bd1862f24466a3', 'request_id': 'req-f9c3d891-961e-411f-97f8-57e242a1cc12'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.542839 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944e4404007045db97bd1862f24466a3', 'request_id': 'req-f9c3d891-961e-411f-97f8-57e242a1cc12'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.549004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944e4404007045db97bd1862f24466a3', 'request_id': 'req-f9c3d891-961e-411f-97f8-57e242a1cc12'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.560861 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] The token cache is being invalidate because project 944e4404007045db97bd1862f24466a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:03.561149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.561783 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.563268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1326/2660] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/projects/944e4404007045db97bd1862f24466a3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:03.567557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.569014 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.582265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9c16e-f5fd-4baa-bc4e-8edf12a771cd admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.637311 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.637931 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.638242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.638511 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] PATH_INFO: `/v3/users/ca58808ce10241b897f735e36491db66` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.639655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca58808ce10241b897f735e36491db66)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.641606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:03.687964 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca58808ce10241b897f735e36491db66'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.695646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca58808ce10241b897f735e36491db66'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.696296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca58808ce10241b897f735e36491db66'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.703797 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca58808ce10241b897f735e36491db66'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.715624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-929d4ed9-43d4-47f5-99ac-8eb3732f6e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca58808ce10241b897f735e36491db66'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:03.721962 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1335/2661] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/users/ca58808ce10241b897f735e36491db66 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:03.892642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:03.894153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:03.908551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3d891-961e-411f-97f8-57e242a1cc12 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:03.966319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:03.966901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:03.967179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:03.967453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] PATH_INFO: `/v3/projects/681f4ab2823c4191b048c3eff764cf7d` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:03.968936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=681f4ab2823c4191b048c3eff764cf7d)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:03.970858 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.023516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681f4ab2823c4191b048c3eff764cf7d', 'request_id': 'req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.029042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681f4ab2823c4191b048c3eff764cf7d', 'request_id': 'req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.035319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681f4ab2823c4191b048c3eff764cf7d', 'request_id': 'req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.048291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] The token cache is being invalidate because project 681f4ab2823c4191b048c3eff764cf7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:04.048566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.049228 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-c5bfae6e-ae4b-4af2-85de-3e2dc81537bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.050711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1327/2662] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:03 2022] DELETE /identity/v3/projects/681f4ab2823c4191b048c3eff764cf7d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:04.055526 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.055994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.056106 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.064828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:04.078729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.079919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.141031 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1336/2663] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:04.148931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.149438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.149718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.169378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.170600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.179219 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1328/2664] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:04.183328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.193768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed50f54-d09c-4745-be1e-009845460df3 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.198229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.198815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.199091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.199359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.199992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:04.200291 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.201911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.213174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1337/2665] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:04.217449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.226609 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112dc6c-d109-40bb-869b-d1f75c74086e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.231164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.231746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.232020 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.232292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.232950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.235412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.263354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1329/2666] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:04.268375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.278080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f66b7-756e-41c7-b2a9-84e304ed9bf1 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.332425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.333036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.333313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.333581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.334229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.336125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.359359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1338/2667] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:04.363566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.374982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41961b-ae80-43ed-95d7-da22667e80cc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.380022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.380720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.381070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.381412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.382145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.383871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.389946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:04.396715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1330/2668] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:04.401794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.412474 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc83e20-10b7-4602-862f-87350861dea2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.417208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef4e8f-38ca-4575-bd67-322e233eba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.417804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4e8f-38ca-4575-bd67-322e233eba43 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.418102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4e8f-38ca-4575-bd67-322e233eba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.418358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef4e8f-38ca-4575-bd67-322e233eba43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.418996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef4e8f-38ca-4575-bd67-322e233eba43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.420619 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef4e8f-38ca-4575-bd67-322e233eba43 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.431977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1339/2669] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:04.436745 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.456281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037058f5-b4e4-45e6-934e-40246f7009ad admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.461653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.462341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.462617 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.462918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] PATH_INFO: `/v3/projects/16c2d5d1db0d46ba9fab52c6c18e9fa0/users/5fed48ea7414446983d02c3c72ab465f/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.478573 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c2d5d1db0d46ba9fab52c6c18e9fa0, user_id=5fed48ea7414446983d02c3c72ab465f, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.481270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.495223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1331/2670] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:04 2022] PUT /identity/v3/projects/16c2d5d1db0d46ba9fab52c6c18e9fa0/users/5fed48ea7414446983d02c3c72ab465f/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:04.506379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.506665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.506940 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.536662 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] MFA Rules not processed for user `5fed48ea7414446983d02c3c72ab465f`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.537924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.599139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:04.608240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1340/2671] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:04.642308 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.644071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.713316 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a51d10-3317-4c3d-8d08-9d843d332abe admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.719055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8305ee-380a-4c0f-8613-bf66d049427e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.719866 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8305ee-380a-4c0f-8613-bf66d049427e service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.720207 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8305ee-380a-4c0f-8613-bf66d049427e service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.720692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8305ee-380a-4c0f-8613-bf66d049427e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.730220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8305ee-380a-4c0f-8613-bf66d049427e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.732001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8305ee-380a-4c0f-8613-bf66d049427e service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.741667 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1332/2672] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:04.757950 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:04.820906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d96eb-f5c7-4c2d-869b-200e3c2490e6 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:04.825377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:04.825990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.826265 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.826535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] PATH_INFO: `/v3/users/5fed48ea7414446983d02c3c72ab465f` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.828378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fed48ea7414446983d02c3c72ab465f)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:04.830376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:04.875249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fed48ea7414446983d02c3c72ab465f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.882805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fed48ea7414446983d02c3c72ab465f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.883410 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fed48ea7414446983d02c3c72ab465f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.892326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fed48ea7414446983d02c3c72ab465f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.907496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fed48ea7414446983d02c3c72ab465f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:04.913730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1341/2673] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:04 2022] DELETE /identity/v3/users/5fed48ea7414446983d02c3c72ab465f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:04.918267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:04.918570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:04.918841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:04.937692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:04.938878 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:04.999500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1333/2674] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:05.009399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.010821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.067063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2c0a-0e4e-432e-8896-1b059ae48603 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.070693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe14d8c-59d4-41b5-a454-9d64b07375d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.071286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe14d8c-59d4-41b5-a454-9d64b07375d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.071561 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe14d8c-59d4-41b5-a454-9d64b07375d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.071832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe14d8c-59d4-41b5-a454-9d64b07375d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.081215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe14d8c-59d4-41b5-a454-9d64b07375d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.082631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe14d8c-59d4-41b5-a454-9d64b07375d7 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.091794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1342/2675] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:05.363093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.364717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.385435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec845887-d7ee-43a0-95f2-07c086e5693d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.391644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.392324 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.392602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.392906 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] PATH_INFO: `/v3/projects/16c2d5d1db0d46ba9fab52c6c18e9fa0` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.394150 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c2d5d1db0d46ba9fab52c6c18e9fa0)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.396299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.443269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c2d5d1db0d46ba9fab52c6c18e9fa0', 'request_id': 'req-150d5ebc-00d8-4a3f-852b-f4175eabe261'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.448795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c2d5d1db0d46ba9fab52c6c18e9fa0', 'request_id': 'req-150d5ebc-00d8-4a3f-852b-f4175eabe261'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.455244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c2d5d1db0d46ba9fab52c6c18e9fa0', 'request_id': 'req-150d5ebc-00d8-4a3f-852b-f4175eabe261'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.467236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] The token cache is being invalidate because project 16c2d5d1db0d46ba9fab52c6c18e9fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:05.467543 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.468893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-150d5ebc-00d8-4a3f-852b-f4175eabe261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.470367 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1334/2676] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:05 2022] DELETE /identity/v3/projects/16c2d5d1db0d46ba9fab52c6c18e9fa0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:05.475243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.475530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.475807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.495880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:05.497109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.517268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:05.564240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1343/2677] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:05.572199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.572474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.572776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.580842 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:05.595259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:05.596472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.604546 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1335/2678] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:05.608890 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.618369 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c46c64-409b-4be1-b89a-28f2b9459788 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.623001 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.623586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.623892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.624320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.624804 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:05.625103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.626679 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.638431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1344/2679] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:05.642912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.652056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b971f-653a-48e4-adf8-ffe258d7e9ce None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.657771 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.658358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.658637 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.658948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.659579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.661508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.688473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1336/2680] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:05 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:05.693117 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.703573 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8ac9e-1fb7-4a50-9afe-94b3fc468495 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.758569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.759174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.759453 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.759728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.760357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.762320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.788735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1345/2681] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:05 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:05.789805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.800296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184795e0-fa47-4c18-b62e-e0bddb348642 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.805212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.805920 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.806270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.806676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.807453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.809124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.820382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1337/2682] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:05.824245 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.834619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.836313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:05.837692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2db15-e154-4c65-a6fb-43c7ee3b9c04 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.842389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.843588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.843871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.844156 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.844833 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.846440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.849681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083c9b1-a0cc-4e2c-a099-a761dfb0f82b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.854436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.855054 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.855330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.855603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] PATH_INFO: `/v3/users/5c86fb90e0554b43a6c3df9f81baff6b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.856834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c86fb90e0554b43a6c3df9f81baff6b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.860969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1346/2683] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:05.861123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.862356 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:05.872575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcccaac-06bf-41c4-9390-9ab64b94bd65 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:05.879030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a3806-183b-4f71-969f-5ea986a22181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:05.879668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a3806-183b-4f71-969f-5ea986a22181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.879949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a3806-183b-4f71-969f-5ea986a22181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.880226 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a3806-183b-4f71-969f-5ea986a22181 admin admin] PATH_INFO: `/v3/projects/87a7782594334fca9b0673f681bf3bc6/users/998f62ee1a0942a9953918de8d3f0603/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.898803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a3806-183b-4f71-969f-5ea986a22181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a7782594334fca9b0673f681bf3bc6, user_id=998f62ee1a0942a9953918de8d3f0603, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:05.901498 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a3806-183b-4f71-969f-5ea986a22181 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:05.906951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c86fb90e0554b43a6c3df9f81baff6b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.913989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c86fb90e0554b43a6c3df9f81baff6b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.914902 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c86fb90e0554b43a6c3df9f81baff6b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.917462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1347/2684] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:05 2022] PUT /identity/v3/projects/87a7782594334fca9b0673f681bf3bc6/users/998f62ee1a0942a9953918de8d3f0603/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:05.927045 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c86fb90e0554b43a6c3df9f81baff6b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.927530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.927821 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.928138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.933974 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-3bcc205d-2968-46b2-9f99-d5abf3616bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c86fb90e0554b43a6c3df9f81baff6b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:05.941969 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1338/2685] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:05 2022] DELETE /identity/v3/users/5c86fb90e0554b43a6c3df9f81baff6b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:05.946318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:05.946604 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:05.946882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:05.967476 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:05.969565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.033907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1339/2686] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:06.044570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.046641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.079542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] MFA Rules not processed for user `998f62ee1a0942a9953918de8d3f0603`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:06.080893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.087943 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:06.107621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552fb33-a0f4-4564-8120-6dffae7b6280 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.111939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.112537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.112849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.113121 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.122782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.124482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.130325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:06.134152 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1340/2687] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:06.148731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1348/2688] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:06.153514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.154937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.168402 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e38c0c-7303-47ce-b16f-d42facbd2b29 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.173850 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.174441 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.174716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.175053 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.175694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.177715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.209410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1341/2689] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:06 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:06.210550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.222476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbf0a7-fb18-4d39-b1c4-c91e6fc4380b None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.285909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.286556 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.286832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.287174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.287803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.289695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.313416 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1349/2690] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:06 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:06.317665 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.328805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7c17d-9fb7-4a10-ad69-155d5d7d1509 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.333455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.334146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.334424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.334693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.335354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.337013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.347572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1342/2691] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:06.351449 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.362061 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46584a05-d400-4123-be09-a9d001cd65eb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.366719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.367363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.367768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.368044 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] PATH_INFO: `/v3/projects/af43bd8dfcb24f39af7b75aa7dec10d9/users/df06bc0cfd6648e180289026f463eade/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.383566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af43bd8dfcb24f39af7b75aa7dec10d9, user_id=df06bc0cfd6648e180289026f463eade, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.386219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.406352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1350/2692] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:06 2022] PUT /identity/v3/projects/af43bd8dfcb24f39af7b75aa7dec10d9/users/df06bc0cfd6648e180289026f463eade/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:06.407904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.408697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.409840 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.424647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee49f5-01cb-416a-b88c-9f2adcd0f51f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.474166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324341d-a6aa-4d3d-8a69-573ce47dfc13 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.478902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.479505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.479782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.480050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.480703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.482304 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.483852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.484503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.484867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.485144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] PATH_INFO: `/v3/projects/37059c02219a4e54bdbb4e6ce3ed692e` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.486273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37059c02219a4e54bdbb4e6ce3ed692e)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.488184 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.492902 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1351/2693] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:06.496945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.507196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016653e-ba8d-4739-9be6-a944312a8996 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.512906 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5b5a9-b9b5-4b77-978b-e67024bd72b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.513541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b5a9-b9b5-4b77-978b-e67024bd72b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.513819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b5a9-b9b5-4b77-978b-e67024bd72b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.514092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b5a9-b9b5-4b77-978b-e67024bd72b9 admin admin] PATH_INFO: `/v3/projects/af43bd8dfcb24f39af7b75aa7dec10d9/users/df06bc0cfd6648e180289026f463eade/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.516212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5b5a9-b9b5-4b77-978b-e67024bd72b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af43bd8dfcb24f39af7b75aa7dec10d9, user_id=df06bc0cfd6648e180289026f463eade, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.519255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5b5a9-b9b5-4b77-978b-e67024bd72b9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.531292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1352/2694] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:06 2022] PUT /identity/v3/projects/af43bd8dfcb24f39af7b75aa7dec10d9/users/df06bc0cfd6648e180289026f463eade/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:06.537666 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37059c02219a4e54bdbb4e6ce3ed692e', 'request_id': 'req-21440b50-19a2-4767-8b51-2e3836ecbfc3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.542259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.542994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.543180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37059c02219a4e54bdbb4e6ce3ed692e', 'request_id': 'req-21440b50-19a2-4767-8b51-2e3836ecbfc3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.543292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.550134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37059c02219a4e54bdbb4e6ce3ed692e', 'request_id': 'req-21440b50-19a2-4767-8b51-2e3836ecbfc3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.566704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] The token cache is being invalidate because project 37059c02219a4e54bdbb4e6ce3ed692e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:06.566991 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.567634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.569175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1343/2695] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:06 2022] DELETE /identity/v3/projects/37059c02219a4e54bdbb4e6ce3ed692e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:06.572614 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] MFA Rules not processed for user `df06bc0cfd6648e180289026f463eade`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:06.573113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.573944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.574424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.604233 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:06.615340 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:06.643214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21440b50-19a2-4767-8b51-2e3836ecbfc3 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.646597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1353/2696] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:06.658717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:06.660354 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.706630 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.707256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.707537 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.707814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] PATH_INFO: `/v3/users/b13121c7b5bf4712820eb10457a8306b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.708976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13121c7b5bf4712820eb10457a8306b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.710936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.721424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806e3d7-d40a-4183-8840-354d4f5b2ca9 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:06.725084 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:06.725681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:06.725957 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:06.726228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:06.727966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:06.758334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13121c7b5bf4712820eb10457a8306b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.767387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13121c7b5bf4712820eb10457a8306b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.768022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13121c7b5bf4712820eb10457a8306b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.780717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13121c7b5bf4712820eb10457a8306b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.789134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13121c7b5bf4712820eb10457a8306b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:06.789330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:06.790798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:06.795430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1344/2697] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:06 2022] DELETE /identity/v3/users/b13121c7b5bf4712820eb10457a8306b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:06.800418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1354/2698] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:07.106068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:07.108780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:07.203724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef06c72-59ef-4bbf-8093-88360cf370ce admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:07.208230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:07.208876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.209149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.209416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] PATH_INFO: `/v3/projects/bf80660456984f919172372f245db890` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.210504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf80660456984f919172372f245db890)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:07.212428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:07.214701 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:07.261714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf80660456984f919172372f245db890', 'request_id': 'req-a726a463-55ed-47a9-9403-c581e94913c0'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.267567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf80660456984f919172372f245db890', 'request_id': 'req-a726a463-55ed-47a9-9403-c581e94913c0'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.273704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf80660456984f919172372f245db890', 'request_id': 'req-a726a463-55ed-47a9-9403-c581e94913c0'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.285814 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] The token cache is being invalidate because project bf80660456984f919172372f245db890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:07.286099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.286763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.288222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1345/2699] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:07 2022] DELETE /identity/v3/projects/bf80660456984f919172372f245db890 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:07.292237 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:07.303621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d6c48-2887-4177-b857-8ae602c73450 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:07.358615 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:07.359222 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.359495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.359763 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] PATH_INFO: `/v3/users/79339ee264be46379605742cfa085120` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.360946 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79339ee264be46379605742cfa085120)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:07.362816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:07.384185 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:07.410394 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79339ee264be46379605742cfa085120'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.417138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79339ee264be46379605742cfa085120'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.417818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79339ee264be46379605742cfa085120'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.424988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79339ee264be46379605742cfa085120'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.436842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-31444885-d48a-4407-818e-2395036a7499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79339ee264be46379605742cfa085120'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.442844 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1355/2700] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:07 2022] DELETE /identity/v3/users/79339ee264be46379605742cfa085120 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:07.704424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:07.705931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:07.719323 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726a463-55ed-47a9-9403-c581e94913c0 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:07.777309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:07.777932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.778209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.778481 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] PATH_INFO: `/v3/projects/444abe2be6ed4b79aea6e3a936d2304a` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.779581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444abe2be6ed4b79aea6e3a936d2304a)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:07.782084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:07.826711 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444abe2be6ed4b79aea6e3a936d2304a', 'request_id': 'req-88e80829-7f49-4d08-af5d-403b724a1201'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.833021 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444abe2be6ed4b79aea6e3a936d2304a', 'request_id': 'req-88e80829-7f49-4d08-af5d-403b724a1201'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.839031 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444abe2be6ed4b79aea6e3a936d2304a', 'request_id': 'req-88e80829-7f49-4d08-af5d-403b724a1201'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.851691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] The token cache is being invalidate because project 444abe2be6ed4b79aea6e3a936d2304a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:07.851978 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.852594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-88e80829-7f49-4d08-af5d-403b724a1201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:07.854092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1346/2701] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:07 2022] DELETE /identity/v3/projects/444abe2be6ed4b79aea6e3a936d2304a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:07.858947 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.859171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.859447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.878514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:07.879696 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:07.937812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1356/2702] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:07.945758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.946040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.946313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.965773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:07.966966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:07.974983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1347/2703] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:07.979162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:07.989260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fca9f5-3c74-4e1f-bd7b-ebc722907fbc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:07.993724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:07.994308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:07.994581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:07.994851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:07.995483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:07.995777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:07.997396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.009217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1357/2704] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:08.013351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:08.022803 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4dfbc-3908-42c7-a5ff-6d3f1806aeac None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:08.028032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:08.028622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.028930 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.029200 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.029830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:08.031963 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.047148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:08.060835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1348/2705] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:08 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:08.065907 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:08.075659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b2d3f-b967-462d-82b5-97c7b50c5233 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:08.130445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:08.131040 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.131317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.131586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.132213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:08.134142 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.156569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1358/2706] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:08 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:08.160754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:08.171741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46775b1-770f-4a23-aaf4-ed212e17b2fc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:08.176605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:08.177088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.177347 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.177640 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.178272 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:08.179860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.192780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1349/2707] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:08.194131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:08.204975 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4af4a-cc0a-40a5-be08-3b32bcdce9b2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:08.210019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7c404-b40b-4fba-afaf-b30ae2a02e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:08.210738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7c404-b40b-4fba-afaf-b30ae2a02e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.211082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7c404-b40b-4fba-afaf-b30ae2a02e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.211361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7c404-b40b-4fba-afaf-b30ae2a02e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.212021 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7c404-b40b-4fba-afaf-b30ae2a02e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:08.213694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7c404-b40b-4fba-afaf-b30ae2a02e74 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.226010 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1359/2708] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:08.230232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:08.241114 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afaafc1-e0a8-40cf-a7d4-17b2f9ab7b3f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:08.245976 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:08.246625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.246898 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.247224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] PATH_INFO: `/v3/projects/bd106308bfdc4ddea1e1373f0f676252/users/9a89fb3b59e64c828fca617242df2e2b/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.263334 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd106308bfdc4ddea1e1373f0f676252, user_id=9a89fb3b59e64c828fca617242df2e2b, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:08.266263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.281027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1350/2709] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:08 2022] PUT /identity/v3/projects/bd106308bfdc4ddea1e1373f0f676252/users/9a89fb3b59e64c828fca617242df2e2b/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:08.291489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.291803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.292075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.301485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:08.327951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] MFA Rules not processed for user `9a89fb3b59e64c828fca617242df2e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:08.329279 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:08.399242 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1360/2710] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:08.412212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:08.413953 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:08.472333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b2e9b-8326-4810-ac97-8157bd9b5204 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:08.476404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f826a-e90c-42f0-81c2-bdd7baf9fcbe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:08.477029 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f826a-e90c-42f0-81c2-bdd7baf9fcbe service glance] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:08.477305 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f826a-e90c-42f0-81c2-bdd7baf9fcbe service glance] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:08.477578 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f826a-e90c-42f0-81c2-bdd7baf9fcbe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:08.486648 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f826a-e90c-42f0-81c2-bdd7baf9fcbe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:08.488079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f826a-e90c-42f0-81c2-bdd7baf9fcbe service glance] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:08.497767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1351/2711] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:09.402708 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:09.404102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:09.469620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a833e-38e2-45d2-b012-ce6d9d808498 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:09.474752 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:09.475380 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:09.475660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:09.475935 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] PATH_INFO: `/v3/users/998f62ee1a0942a9953918de8d3f0603` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:09.477080 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=998f62ee1a0942a9953918de8d3f0603)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:09.479000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:09.526874 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998f62ee1a0942a9953918de8d3f0603'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:09.535037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998f62ee1a0942a9953918de8d3f0603'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:09.535657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998f62ee1a0942a9953918de8d3f0603'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:09.542861 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998f62ee1a0942a9953918de8d3f0603'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:09.555357 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998f62ee1a0942a9953918de8d3f0603'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:09.562130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1361/2712] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:09 2022] DELETE /identity/v3/users/998f62ee1a0942a9953918de8d3f0603 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:09.566901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:09.567418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:09.567418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:09.586197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:09.587377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:09.648098 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1352/2713] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:09.658571 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:09.659969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:09.676258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:09.720569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab056c19-c5df-4a85-a548-edfe60f698d3 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:09.724234 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:09.724892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:09.725174 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:09.725444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:09.734895 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:09.736319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:09.745677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1362/2714] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:10.039478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.041140 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.054602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c306109-c9e5-478b-aff3-ee3b5197f7d5 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.059960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.060887 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.061170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.062040 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] PATH_INFO: `/v3/projects/87a7782594334fca9b0673f681bf3bc6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.063972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a7782594334fca9b0673f681bf3bc6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.066420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.117271 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a7782594334fca9b0673f681bf3bc6', 'request_id': 'req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.121914 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:10.123123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a7782594334fca9b0673f681bf3bc6', 'request_id': 'req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.132487 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a7782594334fca9b0673f681bf3bc6', 'request_id': 'req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.145750 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] The token cache is being invalidate because project 87a7782594334fca9b0673f681bf3bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:10.146033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.146653 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.148273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1353/2715] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:10 2022] DELETE /identity/v3/projects/87a7782594334fca9b0673f681bf3bc6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:10.152426 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.162326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e454d3-e09a-4964-b560-ef254778997c service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.220925 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.221599 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.221885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.222163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] PATH_INFO: `/v3/users/df06bc0cfd6648e180289026f463eade` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.223392 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=df06bc0cfd6648e180289026f463eade)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.225403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.276700 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df06bc0cfd6648e180289026f463eade'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.283904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df06bc0cfd6648e180289026f463eade'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.284540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df06bc0cfd6648e180289026f463eade'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.291873 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df06bc0cfd6648e180289026f463eade'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.307982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-6c66db33-1bb3-48ff-9e14-5afac87aa3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df06bc0cfd6648e180289026f463eade'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.314892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1363/2716] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:10 2022] DELETE /identity/v3/users/df06bc0cfd6648e180289026f463eade => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:10.590116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.591724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.606646 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6c3b-f828-4a0c-adaf-939925a83dfc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.664658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.665224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.665501 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.665776 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] PATH_INFO: `/v3/projects/af43bd8dfcb24f39af7b75aa7dec10d9` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.666900 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] RBAC: Authorizing `identity:delete_project(project_id=af43bd8dfcb24f39af7b75aa7dec10d9)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.668891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.720594 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af43bd8dfcb24f39af7b75aa7dec10d9', 'request_id': 'req-151b9fc7-20f4-4656-a433-c15bf078826f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.725989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af43bd8dfcb24f39af7b75aa7dec10d9', 'request_id': 'req-151b9fc7-20f4-4656-a433-c15bf078826f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.732452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af43bd8dfcb24f39af7b75aa7dec10d9', 'request_id': 'req-151b9fc7-20f4-4656-a433-c15bf078826f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.746371 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] The token cache is being invalidate because project af43bd8dfcb24f39af7b75aa7dec10d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:10.746724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.747278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-151b9fc7-20f4-4656-a433-c15bf078826f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:10.748772 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1354/2717] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:10 2022] DELETE /identity/v3/projects/af43bd8dfcb24f39af7b75aa7dec10d9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:10.754882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.755373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.755490 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.775785 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:10.777063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.808259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:10.840596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1364/2718] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:10.849733 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.850068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.850342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.869990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:10.871182 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:10.878989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1355/2719] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:10.883391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.894122 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9fcd8-1a95-4e15-99db-30f06c7d9449 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.898779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.899375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.899655 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.899930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.900572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:10.900913 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.902504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.915092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1365/2720] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:10.919489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.930059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5da37c-47ca-4d0f-8593-19c2306966c3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:10.934773 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:10.935378 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:10.935649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:10.935921 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:10.936581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:10.938559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:10.966680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1356/2721] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:10.971769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:10.985904 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d12e2-ff8d-48e9-8bde-6976e1730f14 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.048949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.049582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.049889 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.050159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.050803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.052713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.076578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1366/2722] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:10 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:11.078005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.091004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd697936-644e-440a-8753-8f8fd17c0d76 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.096709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.097304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.097575 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.097844 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.098478 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.100082 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.110647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1357/2723] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:11.115168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.125219 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59708053-ceed-4aae-8a2c-1576d4f28a3f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.130606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6a140-55a8-4389-b45f-9c911539b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.131195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6a140-55a8-4389-b45f-9c911539b8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.131472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6a140-55a8-4389-b45f-9c911539b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.131744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6a140-55a8-4389-b45f-9c911539b8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.132377 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6a140-55a8-4389-b45f-9c911539b8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.134015 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6a140-55a8-4389-b45f-9c911539b8a6 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.145639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1367/2724] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:11.149905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.160085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a7617-0f7b-4053-97e0-03a9d65cab1f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.164718 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.165414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.165636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.165910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] PATH_INFO: `/v3/projects/84fb17d896e344aa8c871bf741305304/users/d4d5d6a240fc4dfba9bc0ced8db6d1e8/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.181173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=84fb17d896e344aa8c871bf741305304, user_id=d4d5d6a240fc4dfba9bc0ced8db6d1e8, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.184243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.198223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1358/2725] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:11 2022] PUT /identity/v3/projects/84fb17d896e344aa8c871bf741305304/users/d4d5d6a240fc4dfba9bc0ced8db6d1e8/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:11.203275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd44a9-4827-4119-97ec-569e9ef04eaf None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.204170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd44a9-4827-4119-97ec-569e9ef04eaf None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.204442 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd44a9-4827-4119-97ec-569e9ef04eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.223899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-21cd44a9-4827-4119-97ec-569e9ef04eaf None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:11.225141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-21cd44a9-4827-4119-97ec-569e9ef04eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.285301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1368/2726] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:11.295092 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:11.296849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:11.310415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:11.357570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df24939-330d-4cf8-a952-9b91f09377aa admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:11.363622 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:11.364258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:11.364530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:11.364833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:11.374270 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:11.376203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:11.385780 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1359/2727] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:15.493653 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.494154 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.494323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.536425 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] MFA Rules not processed for user `d4d5d6a240fc4dfba9bc0ced8db6d1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:15.537590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:15.569820 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:15.612159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1369/2728] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:15.620232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.635056 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2d858-82f0-4075-a4a8-5be815da7738 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.636957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.637566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.637838 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.638108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.638736 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.640862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.671239 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1360/2729] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:15 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:15.677817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.688648 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb52028-5625-4dda-8841-85753d988d27 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.750484 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.751090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.751365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.751635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.752260 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.754456 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.777627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1370/2730] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:15 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:15.782454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.798439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaf7dc-2668-4389-aa87-41eafeb686c7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.804260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.805327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.806530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.806945 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.807737 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.818187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.833590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1361/2731] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:15.840022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.851547 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc040-07ab-4eed-8fd8-c681879cd904 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.856301 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.856993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.857267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.857539 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] PATH_INFO: `/v3/projects/ba240431e4ff4d34a9e2abd7738efb58/users/bf6eca075947460b90e02195311f5ea1/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.873807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba240431e4ff4d34a9e2abd7738efb58, user_id=bf6eca075947460b90e02195311f5ea1, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.876550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.895230 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1371/2732] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:15 2022] PUT /identity/v3/projects/ba240431e4ff4d34a9e2abd7738efb58/users/bf6eca075947460b90e02195311f5ea1/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:15.899417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:15.911392 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4849ebe-1199-4acc-9fab-676072ec6152 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:15.980225 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96306d3c-8825-49ac-82d2-516dac635c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:15.980875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96306d3c-8825-49ac-82d2-516dac635c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:15.981154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96306d3c-8825-49ac-82d2-516dac635c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:15.981427 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96306d3c-8825-49ac-82d2-516dac635c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:15.982076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96306d3c-8825-49ac-82d2-516dac635c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:15.983682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96306d3c-8825-49ac-82d2-516dac635c14 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:15.994359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1362/2733] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:15.998465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:16.012845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c5f10-a7be-4519-b17d-0ccaaca75a4b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:16.018280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:16.019049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:16.019395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:16.019734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] PATH_INFO: `/v3/projects/ba240431e4ff4d34a9e2abd7738efb58/users/bf6eca075947460b90e02195311f5ea1/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:16.021832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba240431e4ff4d34a9e2abd7738efb58, user_id=bf6eca075947460b90e02195311f5ea1, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:16.024568 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:16.036982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1372/2734] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:15 2022] PUT /identity/v3/projects/ba240431e4ff4d34a9e2abd7738efb58/users/bf6eca075947460b90e02195311f5ea1/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:20.679359 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60549e1d-37b2-4806-bf4e-7f79edce30fd None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:20.679909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60549e1d-37b2-4806-bf4e-7f79edce30fd None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:20.680192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60549e1d-37b2-4806-bf4e-7f79edce30fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:20.744214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-60549e1d-37b2-4806-bf4e-7f79edce30fd None None] MFA Rules not processed for user `bf6eca075947460b90e02195311f5ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:20.746283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-60549e1d-37b2-4806-bf4e-7f79edce30fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:20.854996 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1363/2735] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:20.863702 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:20.865532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:20.999155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d12a7-629f-439c-a1f2-1960714999ef admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.021514 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82ecac0d9cc54fefbca5e3b6d81c31b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.022315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.022631 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.022899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.050077 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:21.112839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.114403 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.125182 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1373/2736] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:21.277852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.278136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.278419 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.285345 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:21.339815 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:21.341072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:21.411657 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1364/2737] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:15:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:21.460277 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:21.461684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:21.551560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec91e-15da-4d5a-b154-6d24b7402809 service glance] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:21.556454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:21.557090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:21.557371 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:21.557666 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:21.566399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:21.567963 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:21.576424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1374/2738] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:15:23.795408 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:23.900014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e26029-9da2-4959-aadb-0c6140b3b566 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:23.906635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:23.907779 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:23.908434 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:23.908965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:23.914952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:23.934658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:23.939351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48243b22-bd0b-4694-8889-4dbefe629f95 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:23.963127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1365/2739] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:29.821664 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:29.823876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:29.840097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e5bf7-2a82-482a-8354-36476a9ade40 service swift] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:29.844497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:29.845563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:29.845840 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:29.846110 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:29.847982 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:29.930897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:29.932424 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:29.945527 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1375/2740] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:32.594100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:32.594581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:32.594686 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:32.618547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] MFA Rules not processed for user `d4d5d6a240fc4dfba9bc0ced8db6d1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:32.619916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:32.627430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1366/2741] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:33.901505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:33.903281 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:33.915620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a481ae8-5393-48c0-b442-95054bf7c727 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:33.920136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:33.920791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:33.921069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:33.921508 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] PATH_INFO: `/v3/users/1dc5f1d34cad437ea15efb98a0a1ca31` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:33.922758 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dc5f1d34cad437ea15efb98a0a1ca31)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:33.924665 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:33.941588 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:33.972522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dc5f1d34cad437ea15efb98a0a1ca31'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:33.979893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dc5f1d34cad437ea15efb98a0a1ca31'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:33.980563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dc5f1d34cad437ea15efb98a0a1ca31'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:33.988482 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dc5f1d34cad437ea15efb98a0a1ca31'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.001201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dc5f1d34cad437ea15efb98a0a1ca31'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.007286 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1376/2742] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:33 2022] DELETE /identity/v3/users/1dc5f1d34cad437ea15efb98a0a1ca31 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:34.177258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.189955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e8a4b-f7d2-4165-b932-2bd1a11832cf None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.252527 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.253216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.253490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.253757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] PATH_INFO: `/v3/projects/c1913d0c899c48ab9a7781a9e45ac4a9` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.255115 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1913d0c899c48ab9a7781a9e45ac4a9)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.257059 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.272134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=100 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:34.285069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.286496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.312655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1913d0c899c48ab9a7781a9e45ac4a9', 'request_id': 'req-deef6459-1e16-4aea-b2bb-e64d1d82689f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.319426 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1913d0c899c48ab9a7781a9e45ac4a9', 'request_id': 'req-deef6459-1e16-4aea-b2bb-e64d1d82689f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.325823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1913d0c899c48ab9a7781a9e45ac4a9', 'request_id': 'req-deef6459-1e16-4aea-b2bb-e64d1d82689f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.345175 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] The token cache is being invalidate because project c1913d0c899c48ab9a7781a9e45ac4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:34.345505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.346196 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-deef6459-1e16-4aea-b2bb-e64d1d82689f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:34.348183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1367/2743] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:34 2022] DELETE /identity/v3/projects/c1913d0c899c48ab9a7781a9e45ac4a9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:34.352531 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d6b4-cff2-4d29-86af-46f137721426 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.352845 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d6b4-cff2-4d29-86af-46f137721426 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.353119 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d6b4-cff2-4d29-86af-46f137721426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.363420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51074d2-72fb-4bf0-acfe-ae1481d9c3e2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.378588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c171d6b4-cff2-4d29-86af-46f137721426 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:34.379729 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c171d6b4-cff2-4d29-86af-46f137721426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.439916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.440819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.441168 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.441506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.443507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.452166 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1368/2744] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:34.456198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.457827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.461386 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.461676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.462028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.467884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1377/2745] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:34.481904 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:34.483099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.491125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1369/2746] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:34.495296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.506746 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6ca1f-cd88-49c8-84ff-614d1bcc018d service placement] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.511252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.511864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.512194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.512469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.513136 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:34.513435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.515638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.527091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1378/2747] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:34.531635 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.542829 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e33c-bb2d-4682-bf7c-21b874763042 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.547949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.548642 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.549022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.549365 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.550222 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.552197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.580243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1370/2748] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:34.584471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.594389 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01688-48d5-4897-8ec1-0bca771c1efb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.602923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:34.654433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.654959 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.655255 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.655516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.656196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.658140 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.680640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1379/2749] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:34.684869 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.696572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647980bd-7ad5-4b30-998a-b31262a6bffe admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.701240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.701855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.702135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.702409 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.703965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.705600 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.715453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1371/2750] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:34.719604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.729553 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f07398-dcf4-4583-8b6b-5c0096346175 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.734595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339fc86-ebbd-4545-9c36-bed5a6f7d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.735177 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339fc86-ebbd-4545-9c36-bed5a6f7d7ba admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.735454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339fc86-ebbd-4545-9c36-bed5a6f7d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.735730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339fc86-ebbd-4545-9c36-bed5a6f7d7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.736360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339fc86-ebbd-4545-9c36-bed5a6f7d7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.737994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339fc86-ebbd-4545-9c36-bed5a6f7d7ba admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.748375 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1380/2751] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:34.753534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.763326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1484f51-7003-44f4-8f7e-3afd66ac432f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.767923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.768556 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.768870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.769147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] PATH_INFO: `/v3/projects/bab29d4ec86e4bc797d9605ebf3454ff/users/ba0127465e5b4967862ab2fcce59bde2/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.783493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab29d4ec86e4bc797d9605ebf3454ff, user_id=ba0127465e5b4967862ab2fcce59bde2, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.786190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:34.799486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1372/2752] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:34 2022] PUT /identity/v3/projects/bab29d4ec86e4bc797d9605ebf3454ff/users/ba0127465e5b4967862ab2fcce59bde2/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:34.809784 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.810068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.810334 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.837745 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] MFA Rules not processed for user `ba0127465e5b4967862ab2fcce59bde2`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:34.838930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:34.902046 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1381/2753] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:34.907326 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:34.946805 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:34.969396 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1b6b0-1e92-4342-9522-318dcdbb9816 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:34.973998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:34.974591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:34.974865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:34.975136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:34.975757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:34.977717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.003342 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1373/2754] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:34 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:35.007465 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.018644 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d30a73-b077-408d-bb46-dbbca7e49a48 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.073884 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.074485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.074767 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.075036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.075668 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.077640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.099438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1382/2755] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:35 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:35.103547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.114493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20e5b9-55e4-4aa6-89ac-5f93edc45ec1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.119052 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.119636 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.119911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.120187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.120854 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.122457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.133909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1374/2756] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:35.138000 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.148199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b90f6d-fd81-40da-a470-6de3387304e4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.152867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.153517 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.153798 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.154072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] PATH_INFO: `/v3/projects/03a9e86510584f64b8f5d5192ea9ec22/users/82363a4ca0e6405ba68798c4601a282d/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.164231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:35.172770 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a9e86510584f64b8f5d5192ea9ec22, user_id=82363a4ca0e6405ba68798c4601a282d, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.175483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.188296 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1383/2757] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:35 2022] PUT /identity/v3/projects/03a9e86510584f64b8f5d5192ea9ec22/users/82363a4ca0e6405ba68798c4601a282d/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:35.192510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.202212 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcca2b0-6774-4100-8c96-ed8625f0e59a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.260181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237665c-7758-4241-a521-2b124d7bbbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.260819 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237665c-7758-4241-a521-2b124d7bbbac admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.261108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237665c-7758-4241-a521-2b124d7bbbac admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.261373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237665c-7758-4241-a521-2b124d7bbbac admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.262012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237665c-7758-4241-a521-2b124d7bbbac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.263601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237665c-7758-4241-a521-2b124d7bbbac admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.273330 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1375/2758] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:35.277455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.288155 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd512cf-c9a8-4ae3-8dc4-8b3c8bf64e39 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.292734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.293367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.293642 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.293912 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] PATH_INFO: `/v3/projects/03a9e86510584f64b8f5d5192ea9ec22/users/82363a4ca0e6405ba68798c4601a282d/roles/8a4cd71b629943809347939c4c950275` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.295818 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a9e86510584f64b8f5d5192ea9ec22, user_id=82363a4ca0e6405ba68798c4601a282d, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.299022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.309019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1384/2759] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:35 2022] PUT /identity/v3/projects/03a9e86510584f64b8f5d5192ea9ec22/users/82363a4ca0e6405ba68798c4601a282d/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:35.319479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.319764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.320067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.347680 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] MFA Rules not processed for user `82363a4ca0e6405ba68798c4601a282d`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:35.348932 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:35.417143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1376/2760] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:35.428887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.430929 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:35.494605 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cd668-d652-4b11-96c6-2fb45d56bc03 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.498319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a53c5-389d-442b-922d-27a9121fd408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.498922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a53c5-389d-442b-922d-27a9121fd408 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.499200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a53c5-389d-442b-922d-27a9121fd408 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.499476 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a53c5-389d-442b-922d-27a9121fd408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.509086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a53c5-389d-442b-922d-27a9121fd408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.510598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a53c5-389d-442b-922d-27a9121fd408 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.520550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1385/2761] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:35.604961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:35.614754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35602376-4f2e-4ac7-9dab-4977ad400707 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:35.618300 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:35.618889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:35.619187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:35.619455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:35.651187 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:35.683017 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:35.683536 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:35.695049 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1377/2762] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:38.809970 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:38.810429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:38.811201 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:38.831801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:38.833072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:38.906827 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1386/2763] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:38.920619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:38.923111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:38.998377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8320f3f-e81c-4b8d-a9b0-9e8d9ade5e06 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:39.004502 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe230d0-48d8-4649-acc9-d8b544a35932 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:39.005149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe230d0-48d8-4649-acc9-d8b544a35932 service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:39.005423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe230d0-48d8-4649-acc9-d8b544a35932 service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:39.005744 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe230d0-48d8-4649-acc9-d8b544a35932 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:39.016149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe230d0-48d8-4649-acc9-d8b544a35932 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:39.017822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe230d0-48d8-4649-acc9-d8b544a35932 service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:39.026134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1378/2764] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:15:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:15:45.780004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:45.781862 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:45.878422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f415a-f6fb-4f67-ab10-f5618a63b4e7 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:45.883681 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:45.884627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:45.885148 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:45.885624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] PATH_INFO: `/v3/users/9a89fb3b59e64c828fca617242df2e2b` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:45.887006 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a89fb3b59e64c828fca617242df2e2b)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:45.889643 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:45.923240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:45.966741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a89fb3b59e64c828fca617242df2e2b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:45.985445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a89fb3b59e64c828fca617242df2e2b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:45.986557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a89fb3b59e64c828fca617242df2e2b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:45.995314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a89fb3b59e64c828fca617242df2e2b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.013231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a89fb3b59e64c828fca617242df2e2b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.020882 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1387/2765] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:45 2022] DELETE /identity/v3/users/9a89fb3b59e64c828fca617242df2e2b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:46.025474 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.026541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.026802 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.059923 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:46.061267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.160804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1379/2766] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:46.167691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.169359 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.241550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c47132-6c0d-429d-a89a-5c830eea4a82 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.246381 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b5b7a-7bd2-4f65-8fb2-cec9f974da17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.247454 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b5b7a-7bd2-4f65-8fb2-cec9f974da17 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.248324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b5b7a-7bd2-4f65-8fb2-cec9f974da17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.249025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b5b7a-7bd2-4f65-8fb2-cec9f974da17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.262976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b5b7a-7bd2-4f65-8fb2-cec9f974da17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.265066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b5b7a-7bd2-4f65-8fb2-cec9f974da17 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.277609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1388/2767] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:46.583591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.585138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.599145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ee026-78a2-40b6-8f0f-9f2e13db528d None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.604103 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.604822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.605122 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.605373 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] PATH_INFO: `/v3/projects/bd106308bfdc4ddea1e1373f0f676252` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.606520 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd106308bfdc4ddea1e1373f0f676252)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.608454 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.661108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd106308bfdc4ddea1e1373f0f676252', 'request_id': 'req-b59669e3-e7bc-485b-aee7-4d72b5912c2a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.667153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:46.669928 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd106308bfdc4ddea1e1373f0f676252', 'request_id': 'req-b59669e3-e7bc-485b-aee7-4d72b5912c2a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.678498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd106308bfdc4ddea1e1373f0f676252', 'request_id': 'req-b59669e3-e7bc-485b-aee7-4d72b5912c2a'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.693463 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] The token cache is being invalidate because project bd106308bfdc4ddea1e1373f0f676252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:46.693808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.694577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-b59669e3-e7bc-485b-aee7-4d72b5912c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:46.696534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1380/2768] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:46 2022] DELETE /identity/v3/projects/bd106308bfdc4ddea1e1373f0f676252 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:46.701133 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.701419 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.701694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.725606 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:46.726914 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.801794 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1389/2769] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:46.873528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.873939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.874209 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.897431 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:46.898810 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:46.908939 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1381/2770] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:46.911839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.927543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ff09e-f1b5-4a83-823a-3b81497a5721 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.934427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.938431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.938431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.938431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.938431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:46.938431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.940866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:46.960189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1390/2771] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:46.964417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:46.976068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfeac9-598f-455d-8acc-d9d17ee64e67 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:46.980755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:46.981366 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:46.981638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:46.981905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:46.982530 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:46.984486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.019809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1382/2772] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:46 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:47.024261 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.035264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e802c2-328b-4b5b-9f61-e824074d3d5c admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.084276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:47.126163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.127162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.127779 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.128275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.129283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.131374 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.164516 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.174721 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1391/2773] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:47 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:47.177504 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87284f-62db-44c5-b380-b663a502e734 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.183085 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.183793 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.184108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.184383 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.185070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.186812 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.212898 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.219903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1383/2774] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:47.225463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fa7a42-9087-4a87-bbc9-e35b9871751e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.230373 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849a9c-426b-40d4-8082-a3f86cc76e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.230984 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849a9c-426b-40d4-8082-a3f86cc76e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.231258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849a9c-426b-40d4-8082-a3f86cc76e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.231545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849a9c-426b-40d4-8082-a3f86cc76e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.232263 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb849a9c-426b-40d4-8082-a3f86cc76e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.234036 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb849a9c-426b-40d4-8082-a3f86cc76e32 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.278385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1392/2775] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:47.285762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.324710 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd3dd8-8da1-4b3a-8871-ce95acb5ed90 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.360911 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.361557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.361835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.362105 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] PATH_INFO: `/v3/projects/b48e7f3a322e47f9aa6224dac6a9e454/users/94bd6c08de8743cd989ed3f88ed2aa9f/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.416722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48e7f3a322e47f9aa6224dac6a9e454, user_id=94bd6c08de8743cd989ed3f88ed2aa9f, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.419459 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.467134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1384/2776] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:47 2022] PUT /identity/v3/projects/b48e7f3a322e47f9aa6224dac6a9e454/users/94bd6c08de8743cd989ed3f88ed2aa9f/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:47.473102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411254a2-2a62-4e00-9202-f7d5e22f32ee None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.474445 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411254a2-2a62-4e00-9202-f7d5e22f32ee None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.474955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411254a2-2a62-4e00-9202-f7d5e22f32ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.516661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-411254a2-2a62-4e00-9202-f7d5e22f32ee None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:47.518225 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-411254a2-2a62-4e00-9202-f7d5e22f32ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:47.618072 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1393/2777] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:47.628462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:47.630544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:47.701672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056c5bf-11f9-4337-8bcd-02d163c68a4c admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:47.706625 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:47.707663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:47.707936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:47.708281 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:47.721439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:47.722903 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:47.744616 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1385/2778] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:48.462340 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.462879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.462879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.509769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] MFA Rules not processed for user `94bd6c08de8743cd989ed3f88ed2aa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:48.511127 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.581809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:48.583768 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.597403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37777af9-1049-4b94-b42e-98bed00986cd service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:48.612003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:48.612931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.613376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.613940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] PATH_INFO: `/v3/users/ba0127465e5b4967862ab2fcce59bde2` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.615416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0127465e5b4967862ab2fcce59bde2)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:48.616750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1394/2779] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:48.618151 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:48.628074 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:48.629636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.681484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0127465e5b4967862ab2fcce59bde2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.690591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0127465e5b4967862ab2fcce59bde2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.691217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0127465e5b4967862ab2fcce59bde2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.699657 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:48.704565 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0127465e5b4967862ab2fcce59bde2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.710273 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:48.712562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6a26d561-d633-4c39-be9a-633d23cccf5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0127465e5b4967862ab2fcce59bde2'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:48.714647 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365856e3-bc01-47a0-bcf8-7b1ff3aaeea8 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:48.719814 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:48.720349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.720626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.722418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1386/2780] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:48 2022] DELETE /identity/v3/users/ba0127465e5b4967862ab2fcce59bde2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:48.722538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.724059 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.725517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.725961 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.726541 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.736872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:48.738430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:48.746570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:48.747777 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.747923 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1395/2781] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:48.816180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1387/2782] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:48.826068 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:48.827501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:48.839467 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d26f6-61e9-4679-b5d2-1fddd7cf63ca service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:48.891055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:48.891744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:48.892030 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:48.892302 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:48.902485 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:48.903967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:48.914609 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1396/2783] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:49.198304 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:49.199755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:49.212456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552238b7-8818-4d37-a25d-52924ab7040a None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:49.217160 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:49.217795 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:49.218071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:49.218346 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] PATH_INFO: `/v3/projects/bab29d4ec86e4bc797d9605ebf3454ff` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:49.219606 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab29d4ec86e4bc797d9605ebf3454ff)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:49.221581 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:49.269792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab29d4ec86e4bc797d9605ebf3454ff', 'request_id': 'req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.275164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab29d4ec86e4bc797d9605ebf3454ff', 'request_id': 'req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.281377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab29d4ec86e4bc797d9605ebf3454ff', 'request_id': 'req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.293288 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] The token cache is being invalidate because project bab29d4ec86e4bc797d9605ebf3454ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:49.293576 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.294197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.295683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1388/2784] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:49 2022] DELETE /identity/v3/projects/bab29d4ec86e4bc797d9605ebf3454ff => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:49.300143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:49.309633 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ebc75-a20d-4f3a-b978-fd83e3fc8c05 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:49.365228 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:49.365857 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:49.366138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:49.366418 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] PATH_INFO: `/v3/users/82363a4ca0e6405ba68798c4601a282d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:49.367538 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=82363a4ca0e6405ba68798c4601a282d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:49.369557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:49.415563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82363a4ca0e6405ba68798c4601a282d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.424131 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82363a4ca0e6405ba68798c4601a282d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.424774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82363a4ca0e6405ba68798c4601a282d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.431513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82363a4ca0e6405ba68798c4601a282d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.443624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-68319fd8-b536-4955-bddb-f41f4f3b4cdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82363a4ca0e6405ba68798c4601a282d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.449367 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1397/2785] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:49 2022] DELETE /identity/v3/users/82363a4ca0e6405ba68798c4601a282d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:49.709830 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:49.711327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:49.741455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f44974b-fb3d-4a16-8a57-f9b5a60f1167 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:49.803712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:49.804391 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:49.804688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:49.804967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] PATH_INFO: `/v3/projects/03a9e86510584f64b8f5d5192ea9ec22` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:49.806084 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a9e86510584f64b8f5d5192ea9ec22)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:49.808005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:49.833291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:49.861503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a9e86510584f64b8f5d5192ea9ec22', 'request_id': 'req-0bf85314-1f80-4238-8dbc-92ccb6493b09'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.867039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a9e86510584f64b8f5d5192ea9ec22', 'request_id': 'req-0bf85314-1f80-4238-8dbc-92ccb6493b09'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.873418 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a9e86510584f64b8f5d5192ea9ec22', 'request_id': 'req-0bf85314-1f80-4238-8dbc-92ccb6493b09'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.886384 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] The token cache is being invalidate because project 03a9e86510584f64b8f5d5192ea9ec22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:49.886672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.887297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0bf85314-1f80-4238-8dbc-92ccb6493b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:49.888782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1389/2786] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:49 2022] DELETE /identity/v3/projects/03a9e86510584f64b8f5d5192ea9ec22 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:49.896455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:49.896990 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:49.897101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:49.905506 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:49.917593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:49.918778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:49.975961 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1398/2787] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:49.983522 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:49.983816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:49.984603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.005358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:50.006560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:50.014525 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1390/2788] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:50.018697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.028152 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647ad19-ac7a-4ff4-aac7-6363a7fb93e4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.032612 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.033231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.033496 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.033765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.034398 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:50.034693 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.036275 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.048130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1399/2789] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:50.052302 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.061382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ba4f4-1df1-435a-a320-def1d2783bf2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.066808 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e0bc2-fb16-4207-86eb-ee5760613637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.067415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0bc2-fb16-4207-86eb-ee5760613637 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.067691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0bc2-fb16-4207-86eb-ee5760613637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.067964 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e0bc2-fb16-4207-86eb-ee5760613637 admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.068589 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e0bc2-fb16-4207-86eb-ee5760613637 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.070515 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e0bc2-fb16-4207-86eb-ee5760613637 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.103714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1391/2790] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:50.105009 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.121566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605abcbb-f91e-4a8c-9929-d16815e6a1c2 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.134424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.134705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.134977 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.154043 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] MFA Rules not processed for user `94bd6c08de8743cd989ed3f88ed2aa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:50.155250 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:50.182108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.182707 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.182987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.183259 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.183875 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.185810 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.207170 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1400/2791] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:50.211141 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.211950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1392/2792] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:50.221323 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd66c4-e42b-47c1-b8d9-eaf29cd00b04 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.225834 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.226415 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.226690 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.226956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.227584 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.229196 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.246809 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1401/2793] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:50.250787 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.262538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa15aa-a249-456e-b6cd-dafb7aec97a6 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.268108 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c5dff-ddc1-44f3-924f-900dab49da14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.268761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c5dff-ddc1-44f3-924f-900dab49da14 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.269039 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c5dff-ddc1-44f3-924f-900dab49da14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.269341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c5dff-ddc1-44f3-924f-900dab49da14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.270014 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c5dff-ddc1-44f3-924f-900dab49da14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.271661 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c5dff-ddc1-44f3-924f-900dab49da14 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.287185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1393/2794] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:50.288326 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.299268 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14658824-00c3-4768-8af2-c08d873adb10 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.304165 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.304879 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.305159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.305433 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] PATH_INFO: `/v3/projects/044ebdb248c2406b8b1e81cbcea91ee6/users/39d31422491a4acf969d42e408737a9e/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.321791 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=044ebdb248c2406b8b1e81cbcea91ee6, user_id=39d31422491a4acf969d42e408737a9e, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.324360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.338469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1402/2795] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:50 2022] PUT /identity/v3/projects/044ebdb248c2406b8b1e81cbcea91ee6/users/39d31422491a4acf969d42e408737a9e/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:50.349353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.349641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.349936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.379958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] MFA Rules not processed for user `39d31422491a4acf969d42e408737a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:50.381291 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:50.456598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1394/2796] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:50.461011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.509706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:50.529202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b586f6-b1ae-4ec4-bd42-f69ded2af7de admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.534358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.535083 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.535362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.535632 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.536251 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.538208 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.568107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1403/2797] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:50.572214 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.579071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:50.583610 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be3949-7621-44ff-b931-41d4b7aec359 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.641009 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.641613 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.641890 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.642162 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.642784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.644751 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.667469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1395/2798] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:50.671988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.682724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4666de4-4e4e-45a8-8d49-9f18cf9a3b4a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.687598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.688194 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.688471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.689130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.689775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.691376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.702342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1404/2799] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:50.707205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.717486 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25cad9-f04a-4cbf-944e-1679b194ba4f admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.722470 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.723134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.723412 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.723687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] PATH_INFO: `/v3/projects/b43d46d635954fb2abc04b36400a1f55/users/c41c55fb03084ab38332b335961ea2e5/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.739450 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43d46d635954fb2abc04b36400a1f55, user_id=c41c55fb03084ab38332b335961ea2e5, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.742747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.756503 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1396/2800] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:50 2022] PUT /identity/v3/projects/b43d46d635954fb2abc04b36400a1f55/users/c41c55fb03084ab38332b335961ea2e5/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:50.760570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.771628 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadbe85-d83d-49ae-9e38-ca1959d9a4d7 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.829649 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b961df-a5ce-417c-b81d-700867de973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.830253 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b961df-a5ce-417c-b81d-700867de973b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.830528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b961df-a5ce-417c-b81d-700867de973b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.830800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b961df-a5ce-417c-b81d-700867de973b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.831435 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b961df-a5ce-417c-b81d-700867de973b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.833063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b961df-a5ce-417c-b81d-700867de973b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.847438 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1405/2801] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:50.848722 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:50.858979 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75252c8f-2741-4014-9462-f2af5dae4478 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:50.864742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:50.865435 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.865714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.866001 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] PATH_INFO: `/v3/projects/b43d46d635954fb2abc04b36400a1f55/users/c41c55fb03084ab38332b335961ea2e5/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.867942 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43d46d635954fb2abc04b36400a1f55, user_id=c41c55fb03084ab38332b335961ea2e5, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:50.881274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:50.901508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1397/2802] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:50 2022] PUT /identity/v3/projects/b43d46d635954fb2abc04b36400a1f55/users/c41c55fb03084ab38332b335961ea2e5/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:50.913024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:50.913555 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:50.913741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:50.962370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] MFA Rules not processed for user `c41c55fb03084ab38332b335961ea2e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:50.963607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.064280 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1406/2803] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:51.078446 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:51.080548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.147960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0189152-7663-46f2-b559-0e1cad80e8a3 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:51.152134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:51.152912 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.153249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.153591 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.168931 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:51.170403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:51.193322 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1398/2804] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:51.359691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:51.361611 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.433091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:51.474883 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c918980-7d08-4876-a92e-ce80132c0bfe None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:51.485729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:51.488272 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:51.488516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:51.488734 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:51.493713 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:51.513725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:51.516431 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdded47-af06-47bb-93ac-a0d71b9030fb service nova] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:51.536256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1407/2805] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:52.688432 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.749242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b382c8-a891-4e3a-acdb-24a7e8b23691 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.753968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.754579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.754855 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.755131 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] PATH_INFO: `/v3/users/39d31422491a4acf969d42e408737a9e` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.756233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39d31422491a4acf969d42e408737a9e)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:52.758217 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:52.759709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:52.806472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39d31422491a4acf969d42e408737a9e'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:52.813954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39d31422491a4acf969d42e408737a9e'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:52.814566 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39d31422491a4acf969d42e408737a9e'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:52.826464 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39d31422491a4acf969d42e408737a9e'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:52.833843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39d31422491a4acf969d42e408737a9e'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:52.839675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1399/2806] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:52 2022] DELETE /identity/v3/users/39d31422491a4acf969d42e408737a9e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:52.845202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.845703 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.845817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:52.864826 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:52.866167 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:52.924063 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1408/2807] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:52.933623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:52.935073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:52.992086 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817efbe-eb05-4146-ba6e-93520efe2364 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:52.995677 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:52.996268 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:52.996544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:52.996862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.005662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.007089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.016951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1400/2808] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:53.340358 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.341921 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:53.359765 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b708f7-6fea-4a11-a95c-ec5369495a74 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.365427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.365942 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.366220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.366513 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] PATH_INFO: `/v3/projects/044ebdb248c2406b8b1e81cbcea91ee6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.367729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] RBAC: Authorizing `identity:delete_project(project_id=044ebdb248c2406b8b1e81cbcea91ee6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.369688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.437629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044ebdb248c2406b8b1e81cbcea91ee6', 'request_id': 'req-24bc686b-270e-4c8f-8655-a05f20b3896f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.445103 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044ebdb248c2406b8b1e81cbcea91ee6', 'request_id': 'req-24bc686b-270e-4c8f-8655-a05f20b3896f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.458205 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044ebdb248c2406b8b1e81cbcea91ee6', 'request_id': 'req-24bc686b-270e-4c8f-8655-a05f20b3896f'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.469472 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] The token cache is being invalidate because project 044ebdb248c2406b8b1e81cbcea91ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:53.469757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.470388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.471871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1409/2809] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:53 2022] DELETE /identity/v3/projects/044ebdb248c2406b8b1e81cbcea91ee6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:53.476032 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:53.486120 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c48dfa-78c5-4b82-aa42-0c5fef112051 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:53.543407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:53.544038 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:53.544321 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:53.544601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] PATH_INFO: `/v3/users/c41c55fb03084ab38332b335961ea2e5` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:53.545763 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41c55fb03084ab38332b335961ea2e5)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:53.548500 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:53.572191 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:53.596775 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41c55fb03084ab38332b335961ea2e5'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.604099 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41c55fb03084ab38332b335961ea2e5'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.604754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41c55fb03084ab38332b335961ea2e5'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.616870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41c55fb03084ab38332b335961ea2e5'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.623641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9fb70c6-9de4-4b24-ac38-fecf3c932747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41c55fb03084ab38332b335961ea2e5'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:53.630091 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1401/2810] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:53 2022] DELETE /identity/v3/users/c41c55fb03084ab38332b335961ea2e5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:54.280738 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.283147 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:54.301774 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc686b-270e-4c8f-8655-a05f20b3896f admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.361213 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:54.380331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.381384 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.381896 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.382463 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] PATH_INFO: `/v3/projects/b43d46d635954fb2abc04b36400a1f55` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.383792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43d46d635954fb2abc04b36400a1f55)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.385943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.447811 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43d46d635954fb2abc04b36400a1f55', 'request_id': 'req-5a07a252-a408-468e-9231-019998b0d81b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:54.458525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43d46d635954fb2abc04b36400a1f55', 'request_id': 'req-5a07a252-a408-468e-9231-019998b0d81b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:54.472134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43d46d635954fb2abc04b36400a1f55', 'request_id': 'req-5a07a252-a408-468e-9231-019998b0d81b'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:54.488871 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] The token cache is being invalidate because project b43d46d635954fb2abc04b36400a1f55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:54.489232 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:54.489979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-5a07a252-a408-468e-9231-019998b0d81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:54.491594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1410/2811] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:54 2022] DELETE /identity/v3/projects/b43d46d635954fb2abc04b36400a1f55 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:54.496599 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.497095 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.497287 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.524190 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:54.526205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:54.626016 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1402/2812] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:54.633976 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.634257 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.634532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.673626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:54.674854 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:54.683949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1411/2813] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:54.689042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.700892 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb6e3f-fca6-4a8f-b214-dceeb59240f7 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.706867 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.707856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.708489 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.709349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.710202 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:54.710716 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.713437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.733577 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1403/2814] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:54.735714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.746383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27e550-d0d6-4f77-bffc-eff6ac6620ac None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.753221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.754179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.754677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.755216 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.756107 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.758310 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.799562 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1412/2815] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:54 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:54.803849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.819282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e1204-bbf9-4ab4-b08f-f4a5d386d333 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.888517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.889311 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.889598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.890782 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.891841 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.893883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.920456 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1404/2816] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:54 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:54.922501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.932997 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08b6cc-fb07-4174-948a-00205acfaa22 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.938332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.939189 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.939469 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.939750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.940411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.942071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.952540 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1413/2817] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:54.957216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:54.967153 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5f2b-1cf8-4450-852c-7a7c8c5cd686 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:54.972088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d4e9c-e671-4b62-aa66-75ec5f408e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:54.972697 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d4e9c-e671-4b62-aa66-75ec5f408e25 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:54.972989 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d4e9c-e671-4b62-aa66-75ec5f408e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:54.973260 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d4e9c-e671-4b62-aa66-75ec5f408e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:54.973919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d4e9c-e671-4b62-aa66-75ec5f408e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:54.975495 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d4e9c-e671-4b62-aa66-75ec5f408e25 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:54.988123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1405/2818] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:54.992654 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.004159 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3577aca-b350-47d9-8545-97f299313f14 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.008916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.009566 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.009845 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.010120 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] PATH_INFO: `/v3/projects/13d60eaeff1c42038b48eb124810b075/users/9b0d8262ca654da5b310dc1ae1ad9e7a/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.025439 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d60eaeff1c42038b48eb124810b075, user_id=9b0d8262ca654da5b310dc1ae1ad9e7a, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.028098 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.042183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1414/2819] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:54 2022] PUT /identity/v3/projects/13d60eaeff1c42038b48eb124810b075/users/9b0d8262ca654da5b310dc1ae1ad9e7a/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:55.053102 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.053405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.053672 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.067430 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:55.083696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] MFA Rules not processed for user `9b0d8262ca654da5b310dc1ae1ad9e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:55.085731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:55.152382 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1406/2820] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:55.156805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.229688 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76adae-a83d-4e2d-90c3-a7f6ba2c3e9b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.236368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.237037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.237324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.237595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.238300 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.240388 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.280541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1415/2821] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:55 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:55.282833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.294364 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1006fa9-55f1-4415-9166-406a24e60a0e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.356256 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.356894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.357177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.357453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.358100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.360724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.386767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1407/2822] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:55 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 17 21:15:55.391091 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.396722 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:55.402683 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92eeb4-fab2-4faf-aa1a-b5c60e2e74af admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.410109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.410733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.411188 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.412217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.412897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.414495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.426186 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1416/2823] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:55.430208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.441183 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a9d45-ea98-4b93-b919-e4c47d2c640b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.446267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.446894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.447171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.447445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] PATH_INFO: `/v3/projects/5ba4f8ed597d46c9b7a7c98edf2fdb8b/users/d81d5f8ccae54daf9b85e4d58a844188/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.463292 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba4f8ed597d46c9b7a7c98edf2fdb8b, user_id=d81d5f8ccae54daf9b85e4d58a844188, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.466002 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.480089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1408/2824] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:55 2022] PUT /identity/v3/projects/5ba4f8ed597d46c9b7a7c98edf2fdb8b/users/d81d5f8ccae54daf9b85e4d58a844188/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:55.484582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.495132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2b364-5db2-49c5-80a6-b2341a5a1d45 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.575560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7428972-0d3e-4951-94d5-667d0bfaac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.576404 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7428972-0d3e-4951-94d5-667d0bfaac9e admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.576697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7428972-0d3e-4951-94d5-667d0bfaac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.576988 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7428972-0d3e-4951-94d5-667d0bfaac9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.577659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7428972-0d3e-4951-94d5-667d0bfaac9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.579267 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7428972-0d3e-4951-94d5-667d0bfaac9e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.589505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1417/2825] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:55.594567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.605096 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027b956-2cfb-49d6-9a28-5e9e6b3fa528 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.610398 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.610572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.610901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.611240 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] PATH_INFO: `/v3/projects/5ba4f8ed597d46c9b7a7c98edf2fdb8b/users/d81d5f8ccae54daf9b85e4d58a844188/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.613265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba4f8ed597d46c9b7a7c98edf2fdb8b, user_id=d81d5f8ccae54daf9b85e4d58a844188, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.616066 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.629466 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1409/2826] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:55 2022] PUT /identity/v3/projects/5ba4f8ed597d46c9b7a7c98edf2fdb8b/users/d81d5f8ccae54daf9b85e4d58a844188/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:55.641660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.641660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.641660 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.674361 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] MFA Rules not processed for user `d81d5f8ccae54daf9b85e4d58a844188`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:55.675569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:55.751646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1418/2827] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:15:55.763698 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:55.766042 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:55.837496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:55.843284 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa0204-04a4-4934-9152-1cbb58c7f877 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:55.848387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9cbd46-1f07-442d-b19f-355746419713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:55.848874 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cbd46-1f07-442d-b19f-355746419713 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:55.849208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cbd46-1f07-442d-b19f-355746419713 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:55.849406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cbd46-1f07-442d-b19f-355746419713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:55.859524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cbd46-1f07-442d-b19f-355746419713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:55.860998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9cbd46-1f07-442d-b19f-355746419713 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:55.871639 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1410/2828] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:56.258583 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:56.319322 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51244eb-27e2-4b23-bb23-983b3bc7a078 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:56.323892 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:56.324507 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:56.324819 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:56.325094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] PATH_INFO: `/v3/users/9b0d8262ca654da5b310dc1ae1ad9e7a` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:56.326198 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0d8262ca654da5b310dc1ae1ad9e7a)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:56.328719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:56.341733 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:56.373499 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0d8262ca654da5b310dc1ae1ad9e7a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.380993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0d8262ca654da5b310dc1ae1ad9e7a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.381645 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0d8262ca654da5b310dc1ae1ad9e7a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.388535 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0d8262ca654da5b310dc1ae1ad9e7a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.401070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0d8262ca654da5b310dc1ae1ad9e7a'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.407071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1419/2829] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:56 2022] DELETE /identity/v3/users/9b0d8262ca654da5b310dc1ae1ad9e7a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:56.411468 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:56.411747 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:56.412041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:56.431238 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:56.432451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:56.489559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1411/2830] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:56.499125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:56.500563 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:56.556321 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08325937-b29e-4659-8ab1-de666922b999 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:56.559934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:56.560520 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:56.560832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:56.561100 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:56.570113 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:56.571542 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:56.581864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1420/2831] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:15:56.744493 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:56.746094 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:56.841584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b781-7165-42c3-ba43-d825e1ea84f7 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:56.849267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:56.849999 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:56.850395 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:56.850687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:56.853144 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:56.860537 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:56.862075 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:56.874261 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:56.876709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:56.877927 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5474284-c4fe-42f7-a3a0-268d3800b659 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:56.887694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:56.891807 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1412/2832] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:15:56.892278 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:56.892558 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:56.892993 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] PATH_INFO: `/v3/projects/13d60eaeff1c42038b48eb124810b075` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:56.894258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d60eaeff1c42038b48eb124810b075)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:56.896160 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:56.952352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d60eaeff1c42038b48eb124810b075', 'request_id': 'req-50319877-bef6-4f5f-80ba-73478e677183'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.958719 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d60eaeff1c42038b48eb124810b075', 'request_id': 'req-50319877-bef6-4f5f-80ba-73478e677183'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.966430 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d60eaeff1c42038b48eb124810b075', 'request_id': 'req-50319877-bef6-4f5f-80ba-73478e677183'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.978428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] The token cache is being invalidate because project 13d60eaeff1c42038b48eb124810b075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:56.978783 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.979550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:56.981143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1421/2833] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:56 2022] DELETE /identity/v3/projects/13d60eaeff1c42038b48eb124810b075 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:56.986055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:56.996299 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7240c-4f96-4164-9423-90ab98d6630b service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:57.052833 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:57.053545 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.053891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.054243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] PATH_INFO: `/v3/users/d81d5f8ccae54daf9b85e4d58a844188` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.055455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81d5f8ccae54daf9b85e4d58a844188)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:57.057439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:57.080410 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:57.108265 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81d5f8ccae54daf9b85e4d58a844188'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.116731 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81d5f8ccae54daf9b85e4d58a844188'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.117387 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81d5f8ccae54daf9b85e4d58a844188'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.130110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81d5f8ccae54daf9b85e4d58a844188'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.137024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-19fde4d3-4ce2-47fd-baf6-f317db47fe88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81d5f8ccae54daf9b85e4d58a844188'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.143644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1413/2834] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:15:56 2022] DELETE /identity/v3/users/d81d5f8ccae54daf9b85e4d58a844188 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:57.410379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:57.411838 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:57.425352 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50319877-bef6-4f5f-80ba-73478e677183 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:57.439443 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:57.486289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:57.486815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.487089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.487383 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] PATH_INFO: `/v3/projects/5ba4f8ed597d46c9b7a7c98edf2fdb8b` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.488462 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba4f8ed597d46c9b7a7c98edf2fdb8b)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:57.490376 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:57.534937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba4f8ed597d46c9b7a7c98edf2fdb8b', 'request_id': 'req-252e02d6-427a-4a44-92a9-0f286623a2b2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.540939 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba4f8ed597d46c9b7a7c98edf2fdb8b', 'request_id': 'req-252e02d6-427a-4a44-92a9-0f286623a2b2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.548602 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba4f8ed597d46c9b7a7c98edf2fdb8b', 'request_id': 'req-252e02d6-427a-4a44-92a9-0f286623a2b2'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.559593 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] The token cache is being invalidate because project 5ba4f8ed597d46c9b7a7c98edf2fdb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:15:57.559876 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.560487 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-252e02d6-427a-4a44-92a9-0f286623a2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:15:57.562848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1422/2835] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:15:57 2022] DELETE /identity/v3/projects/5ba4f8ed597d46c9b7a7c98edf2fdb8b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:57.567957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.568254 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.568548 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.710327 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:57.711549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:57.769528 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1414/2836] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:57.777231 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.777509 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.777776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.798190 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:57.799396 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:57.807379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1423/2837] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:57.811601 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:57.821909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085164-6284-483b-88a5-baf52d7411f2 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:57.827362 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:57.828046 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.828325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.828597 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.829282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:15:57.829596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:57.831203 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:57.843112 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1415/2838] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:15:57.848777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:57.858570 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b47d0b-c037-4bae-ab70-14fa1e419ef4 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:57.865692 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:57.866541 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.866920 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.867294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.868483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:57.870954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:57.902387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1424/2839] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:57 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:57.907950 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:57.918968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb4eee-7d74-49e0-a976-b2583831ce77 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:57.991952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:57.992562 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:57.992882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:57.993157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:57.993790 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:57.995720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.021484 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1416/2840] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:57 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:58.025557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.039390 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5de6-83ec-4fbf-9094-470b6a5b5f72 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.046134 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.047726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.048973 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.049256 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.050143 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.051893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.066620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1425/2841] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:58.067786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.078584 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285c2fa-788c-4aea-ab8a-1a95487b3381 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.083641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8f412-0462-4948-bb0e-74026364ec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.084243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8f412-0462-4948-bb0e-74026364ec49 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.084517 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8f412-0462-4948-bb0e-74026364ec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.084831 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8f412-0462-4948-bb0e-74026364ec49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.085460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8f412-0462-4948-bb0e-74026364ec49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.087076 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8f412-0462-4948-bb0e-74026364ec49 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.097784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1417/2842] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:58.102166 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.113625 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4238-58df-4d6d-b1de-484b998bdb6a admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.118399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.119039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.119330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.119581 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] PATH_INFO: `/v3/projects/97263d9247e54604a00523dbb602587a/users/910c684a19b543d2a4c3384394443aba/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.134550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97263d9247e54604a00523dbb602587a, user_id=910c684a19b543d2a4c3384394443aba, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.137229 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.150790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1426/2843] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:58 2022] PUT /identity/v3/projects/97263d9247e54604a00523dbb602587a/users/910c684a19b543d2a4c3384394443aba/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:58.218415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.218862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.219138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.226420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:58.250726 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] MFA Rules not processed for user `910c684a19b543d2a4c3384394443aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:58.251957 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:58.317724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1418/2844] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:58.322173 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.379200 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:58.406543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d8ae9-41a5-4c97-b6da-49f8292d7fb9 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.411468 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.412071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.412350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.412695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.413303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.415183 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.443864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1427/2845] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:58 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:58.447788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.459197 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9dee03-96f7-442f-a48f-c9e120eeeee3 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.520005 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.520936 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.521218 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.521473 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.522123 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.524050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.545404 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1419/2846] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:58 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:58.549413 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.559787 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b7b52-e6a7-4091-aa6f-ccd181ce158d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.565363 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.565934 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.566212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.566483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.567119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.568801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.581718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1428/2847] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:58.585739 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.595937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3b2fd-bab2-40f7-bb71-f99b6ee4e72a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.600602 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc3f62-d94e-45d5-b423-2e6733641d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.601285 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc3f62-d94e-45d5-b423-2e6733641d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.601559 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc3f62-d94e-45d5-b423-2e6733641d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.601824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc3f62-d94e-45d5-b423-2e6733641d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.602524 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc3f62-d94e-45d5-b423-2e6733641d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.604104 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc3f62-d94e-45d5-b423-2e6733641d7a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.614424 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1420/2848] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:58.618495 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.629283 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05e56b-6641-4bec-bf39-7851d6dedbc0 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.633955 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.634592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.634867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.635135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] PATH_INFO: `/v3/projects/7bb94145a201494da1c7473213d72d70/users/1de9577278d54557b13daa58575e72bc/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.650995 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb94145a201494da1c7473213d72d70, user_id=1de9577278d54557b13daa58575e72bc, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.653885 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.668249 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1429/2849] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:58 2022] PUT /identity/v3/projects/7bb94145a201494da1c7473213d72d70/users/1de9577278d54557b13daa58575e72bc/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:15:58.678870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.679148 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.679417 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.707309 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] MFA Rules not processed for user `1de9577278d54557b13daa58575e72bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:58.708508 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:58.774647 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:58.779836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1421/2850] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:58.784292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.854756 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30fad7-0047-4a24-b916-b12c9a159c06 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.859937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.860689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.860981 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.861252 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.861881 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.863778 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.890943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1430/2851] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:15:58 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:58.894668 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.905113 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4551433-6ea4-4bd3-9d27-c8306073186e None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:58.959406 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:58.959993 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:58.960266 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:58.960538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:58.961178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:58.963127 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:58.983295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1422/2852] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:15:58 2022] POST /identity/v3/users => generated 297 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:15:58.987839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:58.998050 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d941-fa6c-4ecb-b7b7-d4a35b23b7af admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:59.004115 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:59.004740 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:59.005022 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:59.005289 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:59.005922 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:59.007505 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:59.018150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1431/2853] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:15:59.022258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:59.032614 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec76377e-1348-46ce-83a3-7198a3b139ec admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:59.037157 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:59.037789 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:59.038072 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:59.038339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] PATH_INFO: `/v3/projects/9ee4a58a7a124a25a9dfde1d3d69c9be/users/7917060b3d444240bce07cd2a5f27658/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:59.052875 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee4a58a7a124a25a9dfde1d3d69c9be, user_id=7917060b3d444240bce07cd2a5f27658, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:59.055490 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:59.069022 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1423/2854] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:59 2022] PUT /identity/v3/projects/9ee4a58a7a124a25a9dfde1d3d69c9be/users/7917060b3d444240bce07cd2a5f27658/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:59.072916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:59.083720 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d784e-3589-41bc-8e95-413affa27a6b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:59.138090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e91d4e-96e4-4d0b-a1ac-9a6b3f326eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:59.138691 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e91d4e-96e4-4d0b-a1ac-9a6b3f326eda admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:59.138967 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e91d4e-96e4-4d0b-a1ac-9a6b3f326eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:59.139243 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e91d4e-96e4-4d0b-a1ac-9a6b3f326eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:59.139867 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e91d4e-96e4-4d0b-a1ac-9a6b3f326eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:59.142215 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e91d4e-96e4-4d0b-a1ac-9a6b3f326eda admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:59.151803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1432/2855] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 17 21:15:59.155927 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:59.166259 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398085b-0718-44ba-af7a-2da1737cbe8e admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:59.170868 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:59.171511 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:59.171968 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:59.172185 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] PATH_INFO: `/v3/projects/9ee4a58a7a124a25a9dfde1d3d69c9be/users/7917060b3d444240bce07cd2a5f27658/roles/8a4cd71b629943809347939c4c950275` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:59.174243 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ee4a58a7a124a25a9dfde1d3d69c9be, user_id=7917060b3d444240bce07cd2a5f27658, role_id=8a4cd71b629943809347939c4c950275)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:59.176824 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:59.192307 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1424/2856] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:15:59 2022] PUT /identity/v3/projects/9ee4a58a7a124a25a9dfde1d3d69c9be/users/7917060b3d444240bce07cd2a5f27658/roles/8a4cd71b629943809347939c4c950275 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:15:59.202420 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:59.202717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:59.204064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:59.213567 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:59.238887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] MFA Rules not processed for user `7917060b3d444240bce07cd2a5f27658`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:15:59.240125 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:59.331444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1433/2857] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:15:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:15:59.343116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:15:59.345037 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:15:59.421198 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3f10b-ab64-46c9-b1a3-548f7a00ff3b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:15:59.425544 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:15:59.426145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:15:59.426421 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:15:59.426816 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:15:59.492998 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:15:59.494462 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:15:59.505062 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1425/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:01.085595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:01.087124 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:01.098951 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74526d-2564-497a-a7bf-5623ba5b2e9f None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:01.102565 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:01.103163 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:01.103446 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:01.103714 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:01.160109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:01.161560 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:01.170308 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1434/2859] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:01.760235 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:01.770894 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495310ed-f8c4-4fb4-9df4-35b2b3989f95 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:01.775249 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c327-31ac-4ef1-b8af-8549d1c25127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:01.775972 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c327-31ac-4ef1-b8af-8549d1c25127 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:01.776314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c327-31ac-4ef1-b8af-8549d1c25127 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:01.776658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c327-31ac-4ef1-b8af-8549d1c25127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:01.783678 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:01.790551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c327-31ac-4ef1-b8af-8549d1c25127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:01.792568 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c327-31ac-4ef1-b8af-8549d1c25127 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:01.815689 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1426/2860] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:05.755531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:05.818387 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38edd058-b4f3-435f-8e0f-6108360e3d66 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:05.822880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:05.823501 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:05.823776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:05.824048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] PATH_INFO: `/v3/users/910c684a19b543d2a4c3384394443aba` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:05.825319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] RBAC: Authorizing `identity:delete_user(user_id=910c684a19b543d2a4c3384394443aba)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:05.827475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:05.862150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:05.876979 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910c684a19b543d2a4c3384394443aba'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.884319 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910c684a19b543d2a4c3384394443aba'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.885019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910c684a19b543d2a4c3384394443aba'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.892290 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910c684a19b543d2a4c3384394443aba'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.904346 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910c684a19b543d2a4c3384394443aba'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:05.910924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1435/2861] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:05 2022] DELETE /identity/v3/users/910c684a19b543d2a4c3384394443aba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:05.915413 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:05.915883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:05.915980 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:05.936027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:05.937215 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:05.996449 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1427/2862] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:06.007235 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.008652 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.066629 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8034-d4ef-4a5c-aa4e-0d8fb0e0169e admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.070828 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.071451 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.071726 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.071994 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.082195 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.083616 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.092899 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1436/2863] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:06.364767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.366180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.383893 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43319a2-a61f-4204-a72b-bc2389ecf497 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.388750 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.389381 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.389664 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.389990 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] PATH_INFO: `/v3/projects/97263d9247e54604a00523dbb602587a` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.391693 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97263d9247e54604a00523dbb602587a)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.393778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.444742 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97263d9247e54604a00523dbb602587a', 'request_id': 'req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.450136 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97263d9247e54604a00523dbb602587a', 'request_id': 'req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.456918 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97263d9247e54604a00523dbb602587a', 'request_id': 'req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.469632 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] The token cache is being invalidate because project 97263d9247e54604a00523dbb602587a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:06.470055 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.470706 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.472177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1428/2864] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:06 2022] DELETE /identity/v3/projects/97263d9247e54604a00523dbb602587a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:06.476264 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.486247 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2167f2-ad2f-4c0c-ad6f-a19c99a5e9fd service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.543444 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.544067 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.544347 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.544621 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] PATH_INFO: `/v3/users/1de9577278d54557b13daa58575e72bc` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.545764 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de9577278d54557b13daa58575e72bc)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.547878 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:06.597076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de9577278d54557b13daa58575e72bc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.601350 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:06.606661 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de9577278d54557b13daa58575e72bc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.620769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de9577278d54557b13daa58575e72bc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.620769 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de9577278d54557b13daa58575e72bc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.630775 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de9577278d54557b13daa58575e72bc'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:06.636851 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1437/2865] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:06 2022] DELETE /identity/v3/users/1de9577278d54557b13daa58575e72bc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:06.912338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:06.913809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:06.929720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ed4ac-82c8-47ae-91a3-aec5e128e6c1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:06.987727 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:06.988274 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:06.988554 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:06.988856 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] PATH_INFO: `/v3/projects/7bb94145a201494da1c7473213d72d70` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:06.990088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb94145a201494da1c7473213d72d70)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:06.992067 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.014041 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:07.039848 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb94145a201494da1c7473213d72d70', 'request_id': 'req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.045216 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb94145a201494da1c7473213d72d70', 'request_id': 'req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.052178 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb94145a201494da1c7473213d72d70', 'request_id': 'req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.066455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] The token cache is being invalidate because project 7bb94145a201494da1c7473213d72d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:07.066860 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.067623 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.069341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1429/2866] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:06 2022] DELETE /identity/v3/projects/7bb94145a201494da1c7473213d72d70 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:07.073497 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.074937 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.087901 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e31b9-5765-44ba-a47c-22c0d08e6353 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.145687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.146339 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.146624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.146897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] PATH_INFO: `/v3/users/7917060b3d444240bce07cd2a5f27658` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.148026 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7917060b3d444240bce07cd2a5f27658)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.150024 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.197337 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7917060b3d444240bce07cd2a5f27658'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.204747 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7917060b3d444240bce07cd2a5f27658'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.205368 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7917060b3d444240bce07cd2a5f27658'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.212330 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7917060b3d444240bce07cd2a5f27658'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.224594 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-a22a2315-a68f-4095-8ede-5f98de4b8db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7917060b3d444240bce07cd2a5f27658'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.230762 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1438/2867] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:16:07 2022] DELETE /identity/v3/users/7917060b3d444240bce07cd2a5f27658 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:07.512111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.513552 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.526224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5d4aa-63eb-42e4-bdd0-f04b499a0ca3 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.587694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.588317 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.588592 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.588905 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] PATH_INFO: `/v3/projects/9ee4a58a7a124a25a9dfde1d3d69c9be` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.590013 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ee4a58a7a124a25a9dfde1d3d69c9be)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.591916 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.640231 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ee4a58a7a124a25a9dfde1d3d69c9be', 'request_id': 'req-53854918-8d77-4409-8dcb-549521a8abbb'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.646638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ee4a58a7a124a25a9dfde1d3d69c9be', 'request_id': 'req-53854918-8d77-4409-8dcb-549521a8abbb'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.653192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ee4a58a7a124a25a9dfde1d3d69c9be', 'request_id': 'req-53854918-8d77-4409-8dcb-549521a8abbb'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.667224 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] The token cache is being invalidate because project 9ee4a58a7a124a25a9dfde1d3d69c9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:16:07.667512 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.668128 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-53854918-8d77-4409-8dcb-549521a8abbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:16:07.669619 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1430/2868] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:16:07 2022] DELETE /identity/v3/projects/9ee4a58a7a124a25a9dfde1d3d69c9be => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:07.674545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.674830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.675105 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.698807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:07.700041 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.748081 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:07.765489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1439/2869] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:16:07.773891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.774171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.774445 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.795315 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:07.796549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:07.805012 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1431/2870] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:16:07.809365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.820240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bad1c-61eb-4bd5-87f9-b7ab30389d6a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.824911 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.825503 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.825782 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.826057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.826694 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81336) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:16:07.827012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.828938 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.840903 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1440/2871] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 17 21:16:07.845688 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.854888 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4266aec-1d3f-49ff-878e-eae5fe2ec898 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.860089 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.860690 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.860966 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.861230 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.861843 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.863715 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.893173 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1432/2872] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:07.895992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:07.906038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46355f61-d122-49d6-892a-566277a6a5eb admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:07.963483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:07.964090 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:07.964386 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:07.964638 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:07.965303 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:07.967223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:07.991744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1441/2873] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:16:07 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:16:07.993078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.004064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937b722-4bcb-47a9-99df-be4cd45d45fe admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.008557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.009193 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.009469 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.009743 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.010376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.012048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.014129 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:08.027603 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1433/2874] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:08.028776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.039523 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0f4d3-5a26-4f72-9927-dc0cb366d4e0 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.044202 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada58840-b147-4f63-8bde-38c3c180d8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.044837 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada58840-b147-4f63-8bde-38c3c180d8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.045138 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada58840-b147-4f63-8bde-38c3c180d8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.045414 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada58840-b147-4f63-8bde-38c3c180d8c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.046051 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada58840-b147-4f63-8bde-38c3c180d8c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.048314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada58840-b147-4f63-8bde-38c3c180d8c5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.060327 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1442/2875] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:16:08.065242 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.075436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538f1c8-f254-4546-bdfe-5e003f5df28b admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.080050 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.080720 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.081006 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.081282 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] PATH_INFO: `/v3/projects/66a629d38940403aa7ee49b68f1d8ecf/users/21a22c69a4e54c7896fa8e745d1dfc38/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.096959 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66a629d38940403aa7ee49b68f1d8ecf, user_id=21a22c69a4e54c7896fa8e745d1dfc38, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.099876 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.114794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1434/2876] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:16:08 2022] PUT /identity/v3/projects/66a629d38940403aa7ee49b68f1d8ecf/users/21a22c69a4e54c7896fa8e745d1dfc38/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:16:08.119669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c61c4b-66a5-4681-bf9d-594eebc81365 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.120118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c61c4b-66a5-4681-bf9d-594eebc81365 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.120214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c61c4b-66a5-4681-bf9d-594eebc81365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.139427 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-64c61c4b-66a5-4681-bf9d-594eebc81365 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:08.140618 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-64c61c4b-66a5-4681-bf9d-594eebc81365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:08.204637 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1443/2877] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:08.214674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:08.216633 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:08.275213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf6583-30fe-42d8-9cec-40a0d9e1b3e7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:08.278884 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:08.279422 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.279696 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.279965 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.289605 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:08.291048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:08.301794 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1435/2878] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:08.948954 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed529b-7b57-4e49-97f4-866365bdcb28 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:08.949258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed529b-7b57-4e49-97f4-866365bdcb28 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:08.949528 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed529b-7b57-4e49-97f4-866365bdcb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:08.979019 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-87ed529b-7b57-4e49-97f4-866365bdcb28 None None] MFA Rules not processed for user `21a22c69a4e54c7896fa8e745d1dfc38`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:08.980423 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-87ed529b-7b57-4e49-97f4-866365bdcb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:09.027522 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:09.051659 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1444/2879] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:09.062695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:09.064585 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:09.129188 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323483-0c4b-454d-8976-49cfa61d8b0c service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:09.133258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:09.133885 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:09.134318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:09.134634 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:09.144455 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:09.145934 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:09.154687 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1436/2880] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 17 21:16:10.533132 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:10.533743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:10.533743 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:10.553695 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] MFA Rules not processed for user `21a22c69a4e54c7896fa8e745d1dfc38`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:10.554812 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:10.561650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1445/2881] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:10.802134 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:10.811388 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080dec1-1ff3-454b-ad61-3dffe46b8f06 service cinder] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:10.815564 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:10.816206 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:10.816482 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:10.816798 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:10.818682 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:10.831526 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:10.833124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:10.841809 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1437/2882] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:11.252102 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:11.253519 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:11.266924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e682c-3a98-4b06-b4c1-309f9d08ab52 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:11.270604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:11.271199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:11.271483 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:11.271755 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:11.273646 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:11.338293 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:11.339624 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d4897-9ca2-4abe-825b-7b37d7e8816e service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:11.350725 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1446/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:13.207479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:13.209695 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:13.294137 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:13.300035 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5737d66-7569-4690-b765-45d957c70f65 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:13.306649 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:13.308767 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:13.309351 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:13.309702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:13.313024 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:13.332967 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:13.334818 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:13.354377 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1438/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:21.345610 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=40 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:23.345662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:29.472578 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a45953-1014-4599-83b4-5a6595954db8 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:29.473184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a45953-1014-4599-83b4-5a6595954db8 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:29.473184 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a45953-1014-4599-83b4-5a6595954db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:29.492348 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-47a45953-1014-4599-83b4-5a6595954db8 None None] MFA Rules not processed for user `21a22c69a4e54c7896fa8e745d1dfc38`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:29.493607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-47a45953-1014-4599-83b4-5a6595954db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:29.500750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1447/2885] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:16:29.733176 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:16:29.742572 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eab8ef-e010-4747-bf79-4a11bafd8bc2 service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:16:29.746124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:16:29.746709 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:29.746982 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:29.747252 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:29.748951 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:29.758534 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:16:29.759940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eafebb-7842-4471-931c-a932e3b99a09 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:16:29.768498 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1439/2886] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:16:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:16:39.486856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:39.764329 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=6 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:53.201086 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:16:53.201678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:16:53.201678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:16:53.220866 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] MFA Rules not processed for user `91f2a01f833d4751a6f81d6a8f11b8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:16:53.222073 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:16:53.281099 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1448/2887] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:03.271909 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=25 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:08.509741 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7956799-bd60-4772-83cb-d69941e49498 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:08.510277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7956799-bd60-4772-83cb-d69941e49498 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:08.510277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7956799-bd60-4772-83cb-d69941e49498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:08.530093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b7956799-bd60-4772-83cb-d69941e49498 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:08.531269 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b7956799-bd60-4772-83cb-d69941e49498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:08.587262 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1440/2888] 10.222.0.16 () {58 vars in 1009 bytes} [Sat Sep 17 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:08.627640 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:08.629089 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:08.698989 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e364a7e-2904-4bb4-a984-d47875a527fc None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:08.703996 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68415cc-876d-4e96-a91b-74b12e714861 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:08.704596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68415cc-876d-4e96-a91b-74b12e714861 service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:08.705530 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68415cc-876d-4e96-a91b-74b12e714861 service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:08.705808 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68415cc-876d-4e96-a91b-74b12e714861 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:08.713698 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68415cc-876d-4e96-a91b-74b12e714861 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:08.715269 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68415cc-876d-4e96-a91b-74b12e714861 service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:08.723250 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1449/2889] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:17:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:17:13.351353 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:13.351988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:13.351988 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:13.375061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] MFA Rules not processed for user `94bd6c08de8743cd989ed3f88ed2aa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:13.375061 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:13.437919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1441/2890] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:17:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:18.720003 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=32 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:23.429171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=50 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:25.515130 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.515677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.515677 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.535731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:25.536956 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:25.543977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1450/2891] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 17 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:25.556574 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:25.569283 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ee402-678a-42d4-b33e-4c3c3aaa7c96 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:25.576985 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:25.577590 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:25.577864 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:25.578138 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:25.588093 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:25.589880 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:25.599681 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1442/2892] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:17:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:17:27.487315 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.488777 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:27.543834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.557199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f460-03dc-4161-9c13-f3f4ace88a4c service swift] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.562248 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.563264 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.563621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.563895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] PATH_INFO: `/v3/users/21a22c69a4e54c7896fa8e745d1dfc38` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.565090 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a22c69a4e54c7896fa8e745d1dfc38)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.567007 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.569728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb32877-c75a-4973-90c9-26114e38c91a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.578258 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.578795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.579179 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.579521 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.581266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:27.593297 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.596048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.607351 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1451/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:27.618937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a22c69a4e54c7896fa8e745d1dfc38'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.626457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a22c69a4e54c7896fa8e745d1dfc38'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.627485 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a22c69a4e54c7896fa8e745d1dfc38'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.639940 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a22c69a4e54c7896fa8e745d1dfc38'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.646896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-744e52c4-84d5-42f9-a12d-2b1b39c5a99f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a22c69a4e54c7896fa8e745d1dfc38'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.652638 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1443/2894] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:27 2022] DELETE /identity/v3/users/21a22c69a4e54c7896fa8e745d1dfc38 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:27.825582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:27.843057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948430d7-c1e3-4b5a-ab72-444244f07d9a service placement] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:27.900816 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:27.901441 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.901718 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.901992 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] PATH_INFO: `/v3/projects/66a629d38940403aa7ee49b68f1d8ecf` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:27.903097 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] RBAC: Authorizing `identity:delete_project(project_id=66a629d38940403aa7ee49b68f1d8ecf)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:27.905096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:27.950221 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66a629d38940403aa7ee49b68f1d8ecf', 'request_id': 'req-089cf543-63fc-4bb0-92aa-b05d7c5d500d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.956328 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66a629d38940403aa7ee49b68f1d8ecf', 'request_id': 'req-089cf543-63fc-4bb0-92aa-b05d7c5d500d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.963597 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66a629d38940403aa7ee49b68f1d8ecf', 'request_id': 'req-089cf543-63fc-4bb0-92aa-b05d7c5d500d'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.973524 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] The token cache is being invalidate because project 66a629d38940403aa7ee49b68f1d8ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:27.973807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.974457 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-089cf543-63fc-4bb0-92aa-b05d7c5d500d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:27.975893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1452/2895] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:27 2022] DELETE /identity/v3/projects/66a629d38940403aa7ee49b68f1d8ecf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:27.980896 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:27.981179 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:27.981453 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.001702 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:28.002901 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.062796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1444/2896] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:28.071908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.072199 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.072475 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.092411 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:28.093575 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.102320 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1453/2897] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:28.106553 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.116110 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9411-cfb1-43ee-8a5d-268aa58875dd None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.120551 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.121163 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.121436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.121704 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.122355 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:17:28.122655 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.124234 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.135411 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1445/2898] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:17:28.139608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.149270 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1d30c-ab44-4a9b-b0a2-352156dc5891 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.153771 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.154353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.154627 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.154897 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.155516 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.158004 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.162543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:28.186309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1454/2899] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:17:28 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:28.191358 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.201064 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb1f73-fcb2-4630-8e35-5ad93ef4c564 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.243750 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:28.258303 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.258895 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.259170 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.259438 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.260073 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.262015 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.284019 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1446/2900] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:17:28 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:17:28.288101 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.298139 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642db16-a486-4d4f-a8fe-c65b8ece60d4 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.302706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.303287 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.303559 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.303832 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.304459 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.306082 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.316412 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1455/2901] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:28.320169 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.331097 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab3969-265b-4a09-8a8e-5c60b1a99397 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.335659 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3119b8b-ae9c-4eba-8611-177f8fcf6dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.336244 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3119b8b-ae9c-4eba-8611-177f8fcf6dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.336518 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3119b8b-ae9c-4eba-8611-177f8fcf6dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.336828 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3119b8b-ae9c-4eba-8611-177f8fcf6dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.337491 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3119b8b-ae9c-4eba-8611-177f8fcf6dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.339172 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3119b8b-ae9c-4eba-8611-177f8fcf6dcb admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.351181 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1447/2902] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:17:28.355057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.365549 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3244-2fcf-475e-89c0-3bf3971fde6b admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.370043 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.371094 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.371370 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.371639 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] PATH_INFO: `/v3/projects/139547552f26463e8177587b88a94209/users/3efe2bd40cf9424dbb4ad20eb0320ddf/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.386379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=139547552f26463e8177587b88a94209, user_id=3efe2bd40cf9424dbb4ad20eb0320ddf, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.389070 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.402635 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1456/2903] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:17:28 2022] PUT /identity/v3/projects/139547552f26463e8177587b88a94209/users/3efe2bd40cf9424dbb4ad20eb0320ddf/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:17:28.413349 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.413631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.413899 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.441755 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] MFA Rules not processed for user `3efe2bd40cf9424dbb4ad20eb0320ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:28.442946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.505596 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1448/2904] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:28.517378 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.519121 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.574676 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c04f-b143-47c0-84fe-ab3ef2e0d9cd admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.578572 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510f05-9904-4ea3-923d-d009146c3971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.579262 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510f05-9904-4ea3-923d-d009146c3971 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.579532 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510f05-9904-4ea3-923d-d009146c3971 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.579916 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510f05-9904-4ea3-923d-d009146c3971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.588365 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c510f05-9904-4ea3-923d-d009146c3971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.589846 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c510f05-9904-4ea3-923d-d009146c3971 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.598598 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1457/2905] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:28.624488 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.685423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed57e4-65c7-4b27-8fec-9b47f663bef5 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.690941 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.691560 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.691832 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.692100 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] PATH_INFO: `/v3/users/3efe2bd40cf9424dbb4ad20eb0320ddf` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.693236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3efe2bd40cf9424dbb4ad20eb0320ddf)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.695135 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.745692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3efe2bd40cf9424dbb4ad20eb0320ddf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:28.753436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3efe2bd40cf9424dbb4ad20eb0320ddf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:28.754147 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3efe2bd40cf9424dbb4ad20eb0320ddf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:28.756034 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:28.768561 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3efe2bd40cf9424dbb4ad20eb0320ddf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:28.776721 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3efe2bd40cf9424dbb4ad20eb0320ddf'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:28.783663 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1449/2906] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:17:28 2022] DELETE /identity/v3/users/3efe2bd40cf9424dbb4ad20eb0320ddf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:28.788038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.788332 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.788596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.807656 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:17:28.808887 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.866066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1458/2907] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:17:28.875621 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:28.877685 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:28.935983 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8ec3b-91a1-4db9-9bfc-92f7153e9966 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:28.939975 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:28.940569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:28.940883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:28.941154 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:28.950125 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:28.951547 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:28.960954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1450/2908] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:17:29.230276 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:17:29.231715 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:17:29.244607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027a423-8546-45ed-9e8d-38384404cac1 None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:17:29.249529 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:17:29.250191 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:17:29.250471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:17:29.250805 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] PATH_INFO: `/v3/projects/139547552f26463e8177587b88a94209` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:17:29.252037 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=139547552f26463e8177587b88a94209)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:17:29.254039 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:17:29.299240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:29.300790 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139547552f26463e8177587b88a94209', 'request_id': 'req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:29.308880 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139547552f26463e8177587b88a94209', 'request_id': 'req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:29.316379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139547552f26463e8177587b88a94209', 'request_id': 'req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:29.326891 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] The token cache is being invalidate because project 139547552f26463e8177587b88a94209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:17:29.327175 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:29.327807 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:17:29.330096 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1459/2909] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:17:29 2022] DELETE /identity/v3/projects/139547552f26463e8177587b88a94209 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:17:38.956836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=36,keystone:DELETE=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:39.323768 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:19.921820 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:19.923354 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:19.937451 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034f0e-e835-4b92-a1f3-0683e3b9dd41 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:19.994080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:19.994611 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:19.994882 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:19.995213 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] PATH_INFO: `/v3/users/94bd6c08de8743cd989ed3f88ed2aa9f` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:19.996338 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bd6c08de8743cd989ed3f88ed2aa9f)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:19.998277 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.051505 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bd6c08de8743cd989ed3f88ed2aa9f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.062328 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bd6c08de8743cd989ed3f88ed2aa9f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.063428 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bd6c08de8743cd989ed3f88ed2aa9f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.076146 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bd6c08de8743cd989ed3f88ed2aa9f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.083958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-5ac344cf-373a-4f52-8f64-3dc736dda1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bd6c08de8743cd989ed3f88ed2aa9f'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.093952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1451/2910] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:19 2022] DELETE /identity/v3/users/94bd6c08de8743cd989ed3f88ed2aa9f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:20.269941 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.271489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.288092 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4168a7d-e409-4fa9-a6bf-e33c4c8b9fa5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.346212 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.346830 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.347109 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.347382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] PATH_INFO: `/v3/projects/b48e7f3a322e47f9aa6224dac6a9e454` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.348489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48e7f3a322e47f9aa6224dac6a9e454)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.350428 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.397872 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48e7f3a322e47f9aa6224dac6a9e454', 'request_id': 'req-b1045141-d40f-411e-8271-98d6831d6c07'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.404324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48e7f3a322e47f9aa6224dac6a9e454', 'request_id': 'req-b1045141-d40f-411e-8271-98d6831d6c07'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.412144 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48e7f3a322e47f9aa6224dac6a9e454', 'request_id': 'req-b1045141-d40f-411e-8271-98d6831d6c07'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.422434 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] The token cache is being invalidate because project b48e7f3a322e47f9aa6224dac6a9e454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:20.422724 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.423342 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-b1045141-d40f-411e-8271-98d6831d6c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:20.425471 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1460/2911] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:20 2022] DELETE /identity/v3/projects/b48e7f3a322e47f9aa6224dac6a9e454 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:20.430279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.430549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.430823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.450762 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:20.451971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.508960 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1452/2912] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:20.516801 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.517032 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.517309 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.538608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:20.539792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.547687 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1461/2913] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:20.551835 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.561644 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c9eb5-8c7b-4abc-ac34-49445b9d0c8f None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.567071 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.567658 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.567937 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.568208 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.568872 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81335) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 17 21:18:20.569171 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.570759 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.582995 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1453/2914] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Sep 17 21:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 17 21:18:20.587266 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.597402 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6083d-3848-4afd-9c37-238a9a8f8c9a None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.602422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.603011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.603284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.603550 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] PATH_INFO: `/v3/projects` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.604171 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.606057 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.633111 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1462/2915] 10.222.0.16 () {60 vars in 1153 bytes} [Sat Sep 17 21:18:20 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:20.637149 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.648164 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4a669-705e-4189-b8c2-8f3da29217fc admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.651836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:20.706970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.707570 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.707849 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.708124 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.708792 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.710714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.731439 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1454/2916] 10.222.0.16 () {60 vars in 1144 bytes} [Sat Sep 17 21:18:20 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 17 21:18:20.735678 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.745924 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027f456-eba6-434a-b1a6-ec898b6ee95d admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.750543 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.751162 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.751440 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.751741 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.752949 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.754546 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.764730 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1463/2917] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:20.769258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.779079 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680b2b3-93be-41bd-8884-850466898a24 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.783676 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e5726-f283-467f-83b9-967c5d128fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.784257 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5726-f283-467f-83b9-967c5d128fca admin admin] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.784533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5726-f283-467f-83b9-967c5d128fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.784837 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5726-f283-467f-83b9-967c5d128fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.785472 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e5726-f283-467f-83b9-967c5d128fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.787058 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e5726-f283-467f-83b9-967c5d128fca admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.797341 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1455/2918] 10.222.0.16 () {58 vars in 1122 bytes} [Sat Sep 17 21:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 17 21:18:20.801353 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.811066 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cccb78-9ee8-4fe9-a46d-f23570ec6780 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:20.816596 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:20.817739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.818016 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.818285 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] PATH_INFO: `/v3/projects/e85d4fcb64a44804b4e533e41792a116/users/10eb3e8459df4de99d47a3dde175eb08/roles/e5bac5bfcc784719ac8f886a28cb9df6` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.833118 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85d4fcb64a44804b4e533e41792a116, user_id=10eb3e8459df4de99d47a3dde175eb08, role_id=e5bac5bfcc784719ac8f886a28cb9df6)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:20.835739 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:20.848455 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1464/2919] 10.222.0.16 () {60 vars in 1483 bytes} [Sat Sep 17 21:18:20 2022] PUT /identity/v3/projects/e85d4fcb64a44804b4e533e41792a116/users/10eb3e8459df4de99d47a3dde175eb08/roles/e5bac5bfcc784719ac8f886a28cb9df6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:20.859333 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:20.859615 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:20.859889 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:20.888279 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] MFA Rules not processed for user `10eb3e8459df4de99d47a3dde175eb08`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:20.889514 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.952699 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1456/2920] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:20.964817 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:20.966494 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:20.997608 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:21.024930 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a56f91-38d6-418e-9a15-e85141b35581 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:21.028422 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87885a6b-3578-4abe-bd6b-ccb69f69ca55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:21.029038 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87885a6b-3578-4abe-bd6b-ccb69f69ca55 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:21.029313 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87885a6b-3578-4abe-bd6b-ccb69f69ca55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:21.029582 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87885a6b-3578-4abe-bd6b-ccb69f69ca55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:21.037776 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87885a6b-3578-4abe-bd6b-ccb69f69ca55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:21.039203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87885a6b-3578-4abe-bd6b-ccb69f69ca55 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:21.047684 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1465/2921] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:29.294811 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:29.355420 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a0a0-a319-4a6f-a496-cd83f460fb53 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:29.359958 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:29.360569 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:29.360870 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:29.361143 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] PATH_INFO: `/v3/users/10eb3e8459df4de99d47a3dde175eb08` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:29.362786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=10eb3e8459df4de99d47a3dde175eb08)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:29.364724 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:29.380555 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:29.410891 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10eb3e8459df4de99d47a3dde175eb08'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.417954 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10eb3e8459df4de99d47a3dde175eb08'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.418557 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10eb3e8459df4de99d47a3dde175eb08'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.430873 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10eb3e8459df4de99d47a3dde175eb08'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.437717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10eb3e8459df4de99d47a3dde175eb08'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.443822 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1457/2922] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:18:29 2022] DELETE /identity/v3/users/10eb3e8459df4de99d47a3dde175eb08 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:29.448395 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:29.448893 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:29.449007 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:29.467842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] MFA Rules not processed for user `00fcf728ae7844bf9da557cf68143974`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:18:29.469048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:29.527211 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1466/2923] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:18:29.537538 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:29.538919 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:29.594871 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bbffd-04c8-4754-a620-4bdb51468b5a admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:29.598618 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:29.599205 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:29.599479 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:29.599746 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:29.608533 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:29.610003 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:29.619415 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1458/2924] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:29.889314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:29.890757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:29.903349 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33a9b-a317-47e8-8b2e-1b55ac3ea70d None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:29.908399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:29.909064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:29.909858 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:29.910223 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] PATH_INFO: `/v3/projects/e85d4fcb64a44804b4e533e41792a116` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:29.911360 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85d4fcb64a44804b4e533e41792a116)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:29.913314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:29.967095 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85d4fcb64a44804b4e533e41792a116', 'request_id': 'req-188986c1-5dfd-418a-9852-babc8fdd3a36'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.974324 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85d4fcb64a44804b4e533e41792a116', 'request_id': 'req-188986c1-5dfd-418a-9852-babc8fdd3a36'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.981408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85d4fcb64a44804b4e533e41792a116', 'request_id': 'req-188986c1-5dfd-418a-9852-babc8fdd3a36'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.991773 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] The token cache is being invalidate because project e85d4fcb64a44804b4e533e41792a116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:18:29.992145 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.992792 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:18:29.994312 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1467/2925] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:18:29 2022] DELETE /identity/v3/projects/e85d4fcb64a44804b4e533e41792a116 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:18:33.037883 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:18:33.039263 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:33.107295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7e37b-5ef1-4b1f-9c2a-27e36cd92b06 service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:18:33.144186 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:33.166827 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:18:33.167423 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:18:33.167700 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:18:33.167971 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:18:33.169692 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:18:33.181375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:18:33.182956 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:18:33.193651 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1459/2926] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:18:39.988795 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:43.188804 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=14 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:31.769592 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:31.771119 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:31.826908 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188986c1-5dfd-418a-9852-babc8fdd3a36 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:31.830452 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:31.831042 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:31.831362 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:31.832325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:31.834062 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:31.901987 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:31.903477 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:31.914697 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1468/2927] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:36.038339 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:36.039778 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:36.095010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ed4ba-02f2-4edf-bdc0-f024d097251f service placement] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:36.099177 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:36.099758 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:36.100028 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:36.100295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:36.102004 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:36.114862 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:36.116275 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdb70e-3cbe-4667-a2b5-6005ff10b67c service cinder] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:36.126907 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1460/2928] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:40.500864 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:40.511025 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3517b50c-ecb5-498e-bc9a-a632395bcb01 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:40.514620 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f6f37bc49c64ed1a9a61accef657525', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:40.515207 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:40.515481 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:40.515750 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:40.517491 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:40.576841 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:40.578284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:40.587728 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1469/2929] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:41.026836 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] REQUEST_METHOD: `POST` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.027297 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.027407 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.047078 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.auth.core [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] MFA Rules not processed for user `6a042e5b3cc04a83a52b61b1e0bbac33`. Rule list: `[]` (Enabled: `True`). {{(pid=81335) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:41.048258 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:41.102008 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1461/2930] 10.222.0.16 () {58 vars in 967 bytes} [Sat Sep 17 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 17 21:19:41.148071 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:41.149504 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:41.193023 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=100 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:41.218108 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8dd86-2891-44ca-b75a-28f9b8de0911 service cinder] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:41.223135 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e76f823b3a48368f5476c36acb9341', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:41.223735 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:41.224012 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:41.224282 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:41.232203 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:41.233803 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:41.242331 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1470/2931] 10.222.0.16 () {60 vars in 1384 bytes} [Sat Sep 17 21:19:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 17 21:19:44.602549 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:44.604010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:44.622675 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b755141-3eaf-4b24-9cb2-5d38a7d94609 None None] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:44.629192 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:44.629796 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:44.630068 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:44.630337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:44.632087 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:44.644550 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:44.646111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:44.656414 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1462/2932] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:19:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:19:48.072604 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.074055 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.137564 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c95ce-6aba-4a20-9485-29e30b307f01 service swift] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.142157 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.142669 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.142945 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.143757 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] PATH_INFO: `/v3/users/d4d5d6a240fc4dfba9bc0ced8db6d1e8` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.144928 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d5d6a240fc4dfba9bc0ced8db6d1e8)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.146842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.192569 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d5d6a240fc4dfba9bc0ced8db6d1e8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.199974 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d5d6a240fc4dfba9bc0ced8db6d1e8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.200607 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d5d6a240fc4dfba9bc0ced8db6d1e8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.207706 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d5d6a240fc4dfba9bc0ced8db6d1e8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.219960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d5d6a240fc4dfba9bc0ced8db6d1e8'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.226391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1471/2933] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:48 2022] DELETE /identity/v3/users/d4d5d6a240fc4dfba9bc0ced8db6d1e8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:48.404714 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.416010 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a979e5e-bd18-49fa-b352-60ae899ad3ec service placement] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.479460 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.480080 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.480360 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.480631 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] PATH_INFO: `/v3/projects/84fb17d896e344aa8c871bf741305304` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.481757 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84fb17d896e344aa8c871bf741305304)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.483641 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.522723 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:48.532567 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84fb17d896e344aa8c871bf741305304', 'request_id': 'req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.539199 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84fb17d896e344aa8c871bf741305304', 'request_id': 'req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.546047 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84fb17d896e344aa8c871bf741305304', 'request_id': 'req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.559375 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] The token cache is being invalidate because project 84fb17d896e344aa8c871bf741305304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:48.559662 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.560313 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.562452 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1463/2934] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:48 2022] DELETE /identity/v3/projects/84fb17d896e344aa8c871bf741305304 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:48.566385 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.567919 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.581292 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994354d9-f70a-4238-bfd5-dea3e7cfff51 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:48.633151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:48.650379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:48.651126 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:48.651408 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:48.651686 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] PATH_INFO: `/v3/users/bf6eca075947460b90e02195311f5ea1` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:48.652977 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6eca075947460b90e02195311f5ea1)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:48.655056 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:48.713069 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6eca075947460b90e02195311f5ea1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.723049 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6eca075947460b90e02195311f5ea1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.724317 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6eca075947460b90e02195311f5ea1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.733525 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6eca075947460b90e02195311f5ea1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.748151 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-99d62d42-2fc3-4f1c-b627-066235c40ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6eca075947460b90e02195311f5ea1'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:48.754960 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1472/2935] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:19:48 2022] DELETE /identity/v3/users/bf6eca075947460b90e02195311f5ea1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:19:48.936823 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:19:48.938319 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:48.951797 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda7d03-e5ad-479f-b9d4-3d80327b5ee7 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:19:49.011145 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:19:49.011754 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:49.012027 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:49.012295 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] PATH_INFO: `/v3/projects/ba240431e4ff4d34a9e2abd7738efb58` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:49.013405 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba240431e4ff4d34a9e2abd7738efb58)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:19:49.015289 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:19:49.059865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba240431e4ff4d34a9e2abd7738efb58', 'request_id': 'req-63f1dcff-d09c-4871-b35c-c80dc79b70d1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:49.066033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba240431e4ff4d34a9e2abd7738efb58', 'request_id': 'req-63f1dcff-d09c-4871-b35c-c80dc79b70d1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:49.072180 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba240431e4ff4d34a9e2abd7738efb58', 'request_id': 'req-63f1dcff-d09c-4871-b35c-c80dc79b70d1'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:49.084834 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] The token cache is being invalidate because project ba240431e4ff4d34a9e2abd7738efb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81335) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:19:49.085116 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:49.085717 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:19:49.087174 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1464/2936] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:19:48 2022] DELETE /identity/v3/projects/ba240431e4ff4d34a9e2abd7738efb58 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:19:50.645284 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] REQUEST_METHOD: `POST` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:19:50.645815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:19:50.645815 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:19:50.664164 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.auth.core [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] MFA Rules not processed for user `91f2a01f833d4751a6f81d6a8f11b8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81336) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 17 21:19:50.665382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:19:50.720372 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1473/2937] 10.222.0.16 () {58 vars in 958 bytes} [Sat Sep 17 21:19:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 17 21:19:59.081314 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:00.713744 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:07.021852 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:07.023267 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:07.077970 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1dcff-d09c-4871-b35c-c80dc79b70d1 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:07.082088 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:07.082674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:07.082952 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:07.083220 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:07.084946 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:07.145973 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:07.147403 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:07.156785 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1465/2938] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:17.152496 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=50 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:27.818717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:20:27.828150 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7c029-bca7-42cf-b5e7-9959626ad50e None None] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:20:27.832220 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:20:27.832848 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:20:27.833128 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:20:27.833399 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:20:27.835076 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:20:27.844325 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:20:27.846447 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:20:27.854294 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1474/2939] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:20:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:20:37.850717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=6 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:58.439598 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:21:58.441168 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:58.522671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034ed05-ab98-4223-9eb7-2cfabc0e5e0b service neutron] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:21:58.529278 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec86be5855254c8abc10fb636add2749', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:21:58.529994 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:21:58.530318 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:21:58.530588 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:21:58.533155 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:21:58.546447 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:21:58.548909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:21:58.577081 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1466/2940] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:21:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:08.555944 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=34 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:10.678064 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:10.679531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:10.690636 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a2fd-37a9-4b0c-9664-c15fbc698225 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:10.694240 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec498f2822ba4dc9b05a0866e885b5b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:10.694839 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] REQUEST_METHOD: `GET` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:10.695112 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:10.695379 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:10.697053 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:10.757969 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:10.759391 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:10.769944 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1475/2941] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:22:16.911233 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:16.912705 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:16.926436 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb58393-f42f-42d4-95ed-caf2de0d2bbf service nova] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:16.931070 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:16.931683 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:16.931955 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:16.932223 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] PATH_INFO: `/v3/users/91f2a01f833d4751a6f81d6a8f11b8ed` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:16.933325 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91f2a01f833d4751a6f81d6a8f11b8ed)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:16.935492 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:16.986048 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91f2a01f833d4751a6f81d6a8f11b8ed'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.993337 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91f2a01f833d4751a6f81d6a8f11b8ed'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:16.993910 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91f2a01f833d4751a6f81d6a8f11b8ed'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.007579 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91f2a01f833d4751a6f81d6a8f11b8ed'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.014510 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91f2a01f833d4751a6f81d6a8f11b8ed'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.021236 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1467/2942] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:16 2022] DELETE /identity/v3/users/91f2a01f833d4751a6f81d6a8f11b8ed => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 17 21:22:17.195717 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.207856 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48c243-e95d-4856-a22a-b0c22db2f600 service neutron] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.263772 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.264429 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.264731 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.265011 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] PATH_INFO: `/v3/projects/f58304cb0e1a42ce9bbc5ea8a3e58899` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.266630 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f58304cb0e1a42ce9bbc5ea8a3e58899)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.268689 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.315531 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f58304cb0e1a42ce9bbc5ea8a3e58899', 'request_id': 'req-79a42e8c-50f7-4106-aee0-006efc6c4de5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.321842 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f58304cb0e1a42ce9bbc5ea8a3e58899', 'request_id': 'req-79a42e8c-50f7-4106-aee0-006efc6c4de5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.330489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f58304cb0e1a42ce9bbc5ea8a3e58899', 'request_id': 'req-79a42e8c-50f7-4106-aee0-006efc6c4de5'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.340626 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] The token cache is being invalidate because project f58304cb0e1a42ce9bbc5ea8a3e58899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:17.340943 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.341557 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.343052 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1476/2943] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:17 2022] DELETE /identity/v3/projects/f58304cb0e1a42ce9bbc5ea8a3e58899 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:17.347948 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.349416 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:17.362376 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f9f47-cea6-448c-b6b2-5e04b0c8e6e6 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.419457 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.420111 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.420393 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.420691 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] PATH_INFO: `/v3/users/077b25d15fcc42e2b45a44c5f4665e4b` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.421825 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077b25d15fcc42e2b45a44c5f4665e4b)` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.424126 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.471694 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077b25d15fcc42e2b45a44c5f4665e4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.478865 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077b25d15fcc42e2b45a44c5f4665e4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.479521 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077b25d15fcc42e2b45a44c5f4665e4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.492712 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077b25d15fcc42e2b45a44c5f4665e4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.496065 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:17.504437 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.notifications [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077b25d15fcc42e2b45a44c5f4665e4b'} {{(pid=81335) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.510784 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1468/2944] 10.222.0.16 () {58 vars in 1224 bytes} [Sat Sep 17 21:22:17 2022] DELETE /identity/v3/users/077b25d15fcc42e2b45a44c5f4665e4b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:17.681088 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Authenticating user token {{(pid=81336) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:22:17.682729 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.fernet_utils [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81336) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:22:17.698217 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a42e8c-50f7-4106-aee0-006efc6c4de5 admin admin] Validating token access rules against request {{(pid=81336) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:22:17.719685 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:17.757382 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00fcf728ae7844bf9da557cf68143974', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54beae32846044a3b8e04220f82543ad', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81336) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:22:17.757933 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:22:17.758214 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:22:17.758489 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] PATH_INFO: `/v3/projects/3047ba43ac51409aa77e9608816f677d` {{(pid=81336) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:22:17.759595 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3047ba43ac51409aa77e9608816f677d)` {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:22:17.761545 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] RBAC: Authorization granted {{(pid=81336) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:22:17.807314 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3047ba43ac51409aa77e9608816f677d', 'request_id': 'req-ba8e9a4e-86e2-482f-be09-19e585d418ac'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.814590 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3047ba43ac51409aa77e9608816f677d', 'request_id': 'req-ba8e9a4e-86e2-482f-be09-19e585d418ac'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.821622 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3047ba43ac51409aa77e9608816f677d', 'request_id': 'req-ba8e9a4e-86e2-482f-be09-19e585d418ac'} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.832650 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] The token cache is being invalidate because project 3047ba43ac51409aa77e9608816f677d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81336) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 17 21:22:17.832966 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.833586 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG keystone.notifications [None req-ba8e9a4e-86e2-482f-be09-19e585d418ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81336) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 17 21:22:17.835048 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: [pid: 81336|app: 0|req: 1477/2945] 10.222.0.16 () {58 vars in 1233 bytes} [Sat Sep 17 21:22:17 2022] DELETE /identity/v3/projects/3047ba43ac51409aa77e9608816f677d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 17 21:22:27.507232 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=4 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:27.828800 dv-cinder-826510-9-iscsi devstack@keystone.service[81336]: DEBUG dbcounter [-] [81336] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81336) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:37.017761 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Authenticating user token {{(pid=81335) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 17 21:23:37.019201 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:37.155671 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919dd7d-e6db-425f-929c-cfc54106ea16 admin admin] Validating token access rules against request {{(pid=81335) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 17 21:23:37.213788 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8df169f9be474cf6b773aaf4addf0020', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed7f4752773441afbc91d004aa291020', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81335) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 17 21:23:37.214394 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] REQUEST_METHOD: `GET` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 17 21:23:37.214674 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] SCRIPT_NAME: `/identity` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 17 21:23:37.214949 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81335) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 17 21:23:37.216764 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.fernet_utils [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81335) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 17 21:23:37.228296 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 17 21:23:37.229909 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b16d0-915f-4e9e-b366-e18870d48ddb service placement] RBAC: Authorization granted {{(pid=81335) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 17 21:23:37.241033 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: [pid: 81335|app: 0|req: 1469/2946] 10.222.0.16 () {60 vars in 1365 bytes} [Sat Sep 17 21:23:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 17 21:23:47.236786 dv-cinder-826510-9-iscsi devstack@keystone.service[81335]: DEBUG dbcounter [-] [81335] Writing DB stats keystone:SELECT=89 {{(pid=81335) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}